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ABSTRACT 


In  this  dissertation  a  general  method  is  presented 
for  the  structural  analysis  of  natural  texture  images. 
Edge  Repetition  Arrays  (ERAs)  are  calculated  for  the  edge 
and  direction  images  cor r espond ing  to  the  natural  texture 
image  being  analyzed.  Each  ERA  entry  is  the  normalized 
frequency  of  occurrence  of  specific  types  of  edge  matches 
occurring  with  a  particular  angle  and  distance  separation. 
The  two  types  of  edge  matches  sought  are  for  element  size 
and  spacing.  Hence,  a  one-dimensional  structural  texture 
profile  can  be  inferred  from  the  information  inherent  in 
these  arrays.  An  algorithm  designed  to  automatically 
interpret  ERA  results  is  presented.  This  algorithm 
produces  a  one-dimensional  structural  description  of  the 
texture,  and  provides  a  starting  point  for  the 
two-dimensional  texture  primitive  search. 

A  texture  primitive  extraction  algorithms  is  also 
developed.  It  uses  the  information  inherent  in  the 
above-mentioned  texture  description  to  identify  the 
locations  of  various  type  of  elements  within  the  texture 
image.  The  results  produced  are  in  the  form  of  individual 
and  composite  texture  primitive  masks  as  well  as 


descriptions  for  the  individual  texture  primitive  types. 


A  structural  texture  description  should  include 
information  about  the  manner  in  which  the  primitives  are 
arranged  within  the  texture.  Since  the  texture  primitive 
masks  pinpoint  the  texture  element  locations  they  can  be 
used  to  calculate  a  set  of  rules  which  characterize  this 
arrangement.  An  algorithm  is  developed  which  extracts  the 
most  frequently  occurring  interpr imitive  placement  rules, 
and  in  case  the  texture  being  analyzed  is  homogeneous  and 
regular  a  minimum  set  of  rules  necessary  to  characterize 
the  underlying  texture  pattern  is  also  chosen.  Finally,  a 
texture  reconstruction  algorithm  is  developed  to  produce  a 
reconstruction  of  the  original  texture  using  the  minimum 
placement  rule  set  and  a  set  of  average  primitive 
templates.  In  this  way,  a  visual  comparison  can  be  made 
to  determine  how  closely  the  grid  relations  chosen  capture 
the  spatial  relations  of  the  original  texture  image. 

A  scheme  is  presented  for  texture  classification  (of 
11  texture  types)  using  the  structural  texture 
descriptions  discussed  above.  An  overall  success  rate  of 
91.23%  was  achieved.  The  techniques  developed  in  this 
thesis  are  also  applied  to  the  surface  orientation 
determination  problem.  A  general  algorithm  is  outlined 


for  determining  the  orientation  of  textured  surfaces. 
Preliminary  results  are  presented  and  discussed. 


CHAPTER  1 


INTRODUCTION 

When  interpreting  natural  scenes  the  human  visual 
system  uses  the  textural  characteristics  of  the  objects  it 
perceives  as  well  as  their  shapes  and  intensities.  When 
we  differentiate  a  brick  wall  from  a  field  of  grass  or  an 
aerial  ocean  view  from  one  of  a  city  we  are  interpreting 
textural  cues  within  the  image.  A  texture  can  be  thought 
of  as  a  spatial  pattern  of  light  intensities.  This 
pattern,  or  arrangement,  can  be  regular  (periodic)  or 
random.  Likewise,  the  elements  composing  the  texture  can 
be  uniform  or  varying  in  shape,  intensity,  and 
orientation.  Since  texture  is  a  pervasive  characteristic 
of  natural  scenes,  texture  analysis  should  be  included  in 
image  understanding  systems. 

Computer  vision  systems  need  to  have  the  capacity  to 
draw  on  this  wealth  of  textural  information  within  the 
image.  This  information  could  be  helpful  in: 

1)  segmenting  the  image  into  homogeneously  textured 
reg ions, 


1 


2)  providing  more  complete  region  descriptions, 

3)  giving  cues  to  help  predict  surface  orientation;  and, 

4)  aiding  in  region  identification  or  recognition. 

Automatic  analysis  of  naturally  occurring  textures  is 
the  subject  of  this  thesis.  An  attempt  is  made  to 
identify  the  structural  cha r ac te r i st ics  of  the  textures 
being  analyzed.  Descriptions  are  generated  for 
homogeneous  texture  regions,  textural  elements  are 
extracted  and  described,  and  a  set  of  placement  rules 
necessary  for  texture  reconstruction  is  automatically 
determined  whenever  possible. 

1.1  Statistical  Versus  Structural  Analysis  Methods 

A  variety  of  methods  have  been  used  for  texture 
analysis  and  classification.  These  approaches  can,  for 
the  most  part,  be  divided  into  two  major  categories, 
namely,  statistical  and  structural.  In  the  former, 
statistical  measures  are  sought  which  can  be  used  to 
characterize  or  classify  image  textures.  The  experimental 
findings  of  perceptual  psychologist,  Bela  Julesz  [1]  lend 
support  to  this  method  of  analysis.  Julesz  found  that 
textures  whose  first-  or  second-order  statistics  were 
different  could  be  distinguished  immediately  by  human 
observers.  However,  textures  differing  only  in  their 


third-  or  higher-order  statistics  could  not  be  easily 
distinguished.  It  was  therefore  conjectured  that  the 
second-order  statistics  of  a  texture  were  sufficient  for 


texture  characterization.  Later  work  by  others,  including 
Pratt,  Faugeras,  and  Gagalowicz  [2],  has  served  to  sharpen 
this  result.  It  has  been  determined  that  while 
second-order  statistical  measures  should  be  sufficient, 
the  mean,  variance,  and  autocor relation  functions,  by 
themselves,  were  not. 

Structural  analysis  methods,  on  the  other  hand, 
attempt  to  characterize  textures  in  terms  of  primitives 
and  their  relative  placement  within  the  texture  image. 
Texture  primitives  can  be  regions  of  uniform  intensity  or 
more  complex  structures  such  as  tiles  or  mosaic  cells. 

Consider  the  brick  wall  mosaic  in  Fig.  1-1.  The 
statistical  description  of  these  4  brick  subwindows  might 
be  in  the  form  of  feature  vectors  containing  statistical 
measures  designed  to  capture  textural  characteristics  such 
as  contrast  or  coarseness.  (It  is  assumed  that  the  types 
of  measures  chosen  are  nearly  uniform  over  homogeneously 
textured  regions.)  On  the  other  hand,  structural 
descriptions  for  these  4  brick  patterns  include  size, 
shape  and  intensity  information  for  the  brick  and  mortar 
texture  primitives.  They  would  also  provide  descriptions 
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(a)  Brick  Mosaic 
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(b)  Non-Maximal  Suppressed 
Edges  from  (a) 

Figure  1-1.  Brick  Mosaic  and  Edges 
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of  how  these  primitives  are  arranged  in  each  of  the  4 
subpatterns.  Therefore,  it  should  be  clear  from  these 
descriptions  that: 

1)  The  brick  primitives  in  the  top  row  images  are 
arranged  in  rectangular  grid  patterns,  while  the 
images  in  the  bottom  row  show  shifted  arrangements. 

2)  The  horizontal  dimension  of  the  brick  primitives  in 

the  left  top  row  image  is  larger  than  the  vertical 
dimension.  The  opposite  is  true  of  the  brick 

primitives  in  the  right  top  subimage. 

3)  The  brick  primitives  in  the  left  bottom  subimage  are 
smaller  than  the  ones  on  the  right.  (This  information 
could  be  useful  in  determining  the  relative  distance 
from  the  observer.) 

The  two  basic  approaches  need  not  be  used  in 
isolation.  Statistical  methods  have  been  used  to  extract 
structural  information  while  some  structural  textural 
properties  may  need  to  be  described  statistically  [3]. 

1.2  Related  Work 

A  large  variety  of  techniques  have  been  developed  for 
analyzing  natural  textures.  A  few  are  mentioned  briefly 
below.  Some  of  these  techniques  are  discussed  in  greater 


detail  in  later  chapters.  An  excellent  survey  of  texture 
analysis  techniques  can  be  found  in  [4). 


Bajcsy  [5],  (6]  produced  symbolic  descriptions  for 
natural  texture  images  by  analyzing  the  2-dimensional 
Fourier  transform  of  the  image.  By  examining  certain 
regions  within  the  transform  image,  information  pertaining 
to  periodicity,  directionality,  and  other  texture 
characteristics  was  inferred. 

Haralick,  Shanmugam,  and  Dinstein  [7]  developed  a 
technique  for  calculating  certain  textural  features  based 
on  the  gray-tone  spatial  dependencies  inherent  in  texture 
images.  A  set  of  N  x  N  matrices  are  calculated  for  given 
angle  and  distance  pixel  separations  where  N  is  the  number 
of  gray  levels  in  the  image.  The  entries  of  these 
"co-occurrence"  matrices  are  counts  of  how  often  one  gray 
tone  appears  in  a  specific  spatial  relationship  to  another 
gray  tone.  In  [7]  fourteen  measures  are  suggested  for 
calculating  textural  features  from  these  matrices.  (See 
Chapter  2,  Section  2.2  for  some  examples.) 

In  [3]  Price  and  Nevatia  used  the  co-occurrence 
technique  on  edge  images  corresponding  to  natural  texture 
images.  In  this  way  the  number  of  gray  levels  to  consider 
is  only  2  while  the  image  used  retains  the  basic  structure 
of  the  texture  under  consideration. 
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Davis,  Johns,  and  Aggarwal  [9]  generalized  the 
co-occur rence  technique.  Their  generalized  co-occur rence 
matrices  reflect  the  spatial  interdependence  of  sets  of 


local  features  (not  necessarily  gray  levels). 

Clearly,  the  number  of  possible  spatial 
relationships,  and  thus  co-occurrence  matrices,  is  very 
large  for  a  given  texture  image.  In  [10]  Zucker  and 
Terzopoulos  present  a  statistical  approach  to  finding  the 
spatial  relationships  best  suited  to  capture  the 
structural  features  within  a  texture  image. 

Conners  [11]  and  Conners  and  Harlow  [12]  make  use  of 
one  of  the  co-occurrence  measures  to  determine  the  size, 
shape  and  orientation  of  the  unit  cell  or  tile  which  can 
be  repeated  to  produce  a  given  regular  texture.  This  is 
an  example  of  using  a  statistical  texture  analysis  method 
to  extract  structural  information. 

Another  approach  to  texture  analysis  is  the  syntactic 
approach.  It  is  suggested  here  that  the  placement  rules 
for  primitives  (basic  elements  comprising  the  texture)  can 
be  compared  to  the  production  rules  of  a  grammar.  Texture 
classification  is  then  the  determination  of  whether  or  not 
a  particular  texture  image  exhibits  a  pattern  which 
belongs  to  a  given  language.  This  model  was  used  by 
Carlucci  [13],  Lu  and  Fu  [14],  [15],  and  Zucker  [16].  In 
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(16]  Zucker  defines  natural  (observable)  textures  as  ideal 
patterns  which  have  undergone  certain  transformations. 

Primitive  extraction  is  an  important  part  of  the 
structural  texture  analysis  process.  However,  the 
definition  of  "texture  primitive"  is  not  unique. 
Primitives  can  be,  for  example,  line  segments  [13],  pixels 
[14],  or  tiles  [12].  In  [17],  Ehrich  and  Poith  define 
primitives  as  gray  level  peaks.  Then  the  method  of 
relational  trees  is  used  to  recursively  form  more  complex 
regions.  In  [18],  Wang,  Velasco,  Wu,  and  Rosenfeld 
compare  three  threshold  based  primitive  extraction 
techniques.  The  primitives  found  by  these  methods  are 
then  essentially  connected  regions  with  gray  level  values 
restricted  to  a  certain  range  (as  defined  by  the 
threshold) .  In  [19],  Maleson,  Brown,  and  Feldman  describe 
a  texture  analysis  scheme  whereby  the  texture  image  is 
divided  into  uniform  intensity  regions  which  are  then 
approximated  by  ellipses.  These  ellipses  are  their 
texture  primitives. 

Once  the  primitives  are  defined  and  extracted  their 
spatial  arrangements  must  be  ascertained.  In  [19], 
Maleson,  et  al.  discuss  the  types  of  spatial 
relationships  sought  between  elements  of  a  given  class. 
Elements  are  thought  to  be  parallel  or  collinear  depending 


on  how  their  major  axes  align.  In  [20],  Matsuyama, 
Saburi,  and  Nagao  use  regularity  vectors  and  their 
respective  sets  of  element  pairs  to  characterize  commonly 
occurring  inter-primitive  spatial  relations  within  the 
texture.  In  [21],  Davis  presents  a  scheme  for  computing 
the  underlying  grid  relations  for  rectangular  cellular 
textures. 

In  this  thesis  methods  will  be  presented  for: 


1) 

generating 

structur 

al 

descriptions  for 

natural 

textures; 

2) 

extracting 

and  descri 

bing 

textural 

pr imitives; 

and , 

3) 

describing 

inter-pr im 

itive 

spatial 

relations 

within 

the  textur 

e  pattern. 

1.3 

Overv  iew 

The  main 

algor i thms 

presented  in 

this  the: 

sis  are 

outlined  in 

Fig.  1-2. 

The 

texture 

description 

process 

begins  by  creating  edge  and  direction  images  for  the 
texture  image.  Fig.  1-1  provides  examples  of  texture 
images  and  their  corresponding  edge  images.  The  bricks 
and  mortar  strips  are  clearly  outlined,  thus  providing 
primitive  size,  shape,  and  arrangement  information.  Edge 
repetition  arrays  (ERAS)  are  calculated  for  edge  and 
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igure  1»  2.  Structural  Texture  Analysis  Processes 


direction  image  pairs.  An  ERA  entry  is  the  normalized 
frequency  of  occurrence  of  specific  types  of  edge  matches 
occurring  with  a  specified  angle  and  distance  separation 
within  the  texture  image.  Every  picture  element  (pixel) 
within  the  image  is  examined  in  turn.  Starting  at  a  given 
pixel  the  image  is  scanned  in  6  directions  for  a  user 
specified  range  of  distances.  When  a  pair  of  edges  with 
directions  perpendicular  to  the  line  of  scan  are 
encountered  the  matrix  entry  noting  the  particular  type  of 
match  and  spatial  relationship  is  updated. 

There  are  2  types  of  matches  sought.  The  first  type 
of  match  involves  edges  which  define  the  opposite  sides  of 
textural  elements,  e.g.,  opposite  sides  of  the  same  brick 
in  Fig.  1-1 (b) .  In  this  case  the  distance  at  which  the 
match  occurred  would  be  the  dimension  of  the  element  in 
that  scan  direction.  The  second  type  of  match  involves 
the  correspond ing  edges  of  2  neighboring  elements,  e.g., 
the  left  edge  of  2  neighboring  bricks  which  occur  in  the 
same  row  in  Fig.  1-1 (b) .  In  this  case  the  distance  at 
which  the  match  occurred  would  be  the  period  of  repetition 
of  the  texture  elements  in  the  given  scan  direction. 
After  the  ERAs  are  calculated  an  interpretation  routine 
generates  a  texture  description.  This  routine  analyzes 
peaks  and  their  repetitions  in  the  element  size  and 
spacing  ERAs.  The  description  generation  process  is 
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outlined  in  Fig.  1-2  (a)  . 


The  texture  primitive  extraction  process  uses  the 
internal  form  of  the  texture  description  as  its  starting 
point.  The  information  contained  in  the  texture 
description  is  element  size  and  spacing  information  for 
relatively  dark  and  light  texture  elements  found  in  5  scan 
directions.  Using  the  relative  intensity,  size,  and  scan 
direction  for  each  of  the  given  elements  their  exact 
locations  can  be  found  within  the  original  texture  image. 
This  is  done  by  rescanning  in  the  particular  direction  in 
which  the  element  was  found.  However,  the  scan  is  now 
only  for  elements  exhibiting  a  particular  relative 
intensity  and  size. 

When  all  of  the  locations  for  the  element  are  found 
they  are  marked  and  expanded  in  a  blank  primitive  mask 
image.  A  region  extractor  is  then  used  to  examine  the 
primitives  one  by  one  to  determine  some  average 
character istics  of  the  element  type  under  consideration. 
Further  analysis  is  done  to  determine  if  there  is  more 
than  one  primitive  mask  referring  to  the  same  primitive 
type,  or  if  one  mask  refers  to  more  than  one  primitive 
type.  In  these  cases  the  element  masks  are  merged  or 
separated,  respectively.  The  end  product  of  the  primitive 
extraction  process  is  a  set  of  primitive  masks  and 
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descriptions.  (See  Fig.  1-2 (b).) 

Determination  of  inter-primitive  placement  rules  is 
an  important  part  of  the  structural  texture  analysis 
process.  The  primitive  masks  created  by  the  primitive 
extraction  process  are  the  inputs  to  this  algorithm. 

The  centroid  of  each  individual  primitive  is 
computed.  Then  12  directional  scans  are  initiated  from 
each  centroid  location  in  each  primitive  mask.  That  is, 
if  (i,j)  is  the  location  of  the  centroid  of  a  primitive  of 
type  A,  and  there  are  primitives  of  type  A,  B,  and  C  then 
12  directional  scans  are  started  from  location  (i,j)  in 
primitive  masks  A,  B,  and  C.  The  scans  are  continued 
until  a  second  primitive  is  found  or  the  boundary  of  the 
mask  image  is  reached.  When  another  primitive  is 
encountered  the  angle  and  distance  between  the  2  primitive 
centroids  is  noted  and  the  frequency  of  that  particular 
placement  rule  is  increased  by  1.  The  placement  rule 
frequencies  are  then  normalized  and  the  rules  with  the 
highest  rate  of  occurrence  are  kept. 

If  the  texture  being  considered  is  homogeneous  and 
regular  then  a  minimum  set  of  rules  necessary  to 
characterize  the  underlying  grid  pattern  is  extracted. 


This  reduced  set 

o  f 

rules 

along  with  a 

set  of  average 

texture  primitives 

is 

used 

to  create  a 

reconstructed 
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texture  image.  A  visual  comparison  can  then  be  made  to 
determine  how  closely  the  grid  relations  chosen  capture 
the  spatial  relations  of  the  original  texture  image.  The 
spatial  relation  analysis  scheme  is  shown  in  Fig.  1-2  (c) . 

1.4  Organization 

The  organization  of  this  thesis  will  be  discussed 
below.  It  should  be  noted  that  most  of  the  texture 
samples  displayed  and  analyzed  in  this  thesis  are  taken 
from  a  photographic  album  of  texture  images  compiled  by 
Brodatz  [22]. 

In  Chapter  2  a  method  for  automatically  generating 
texture  image  descriptions  is  presented.  First,  the  edge 
repetition  array  (ERA)  and  its  precursor,  the  gray  level 
co-occurrence  matrix,  are  both  discussed  in  detail.  ERA 
results  are  then  displayed  and  interpreted  for  a  number  of 
natural  textures.  (The  information  inherent  in  these 
arrays  is  period  and  element  size  information  in  pixels 
for  a  number  of  specific  scan  directions.)  The  technique 
used  to  automate  the  interpretation  process  is  next 
described,  and  some  automatically  generated  descriptive 
results  are  presented. 

In  Chapter  3  a  method  for  automatically  extracting 
and  describing  textural  elements  is  presented.  First,  an 


"intelligent"  primitive  search  which  uses  the 
one-dimensional  textural  information  extracted  by  the 
algorithm  described  in  Chapter  2  is  explained  in  detail. 
Results  of  this  search  are  in  the  form  of  individual  and 
composite  primitive  image  masks.  Masks  automatically 
generated  for  a  number  of  natural  textures  are  presented 
and  discussed.  The  primitive  analysis  technique  is 
explained  and  the  primitive  descriptions  for  several 
texture  images  are  examined. 

In  Chapter  4  a  method  for  automatically  determining 
inter-primitive  placement  rules  is  presented.  The  role  of 
individual  primitive  masks  in  the  rule  extraction  process 
is  explained  in  detail.  Sets  of  placement  rules  for 
several  periodic  textures  are  examined.  A  technique  for 
extracting  minimum  sets  of  rules  for  completely 
charac ter i zing  homogeneous,  regular  textures  is  examined. 
Then  a  texture  reconstruction  process  using  this  set  of 
rules,  primitive  templates,  and  primitive  descriptions  is 
described.  Reconstruction  results  for  a  number  of 
periodic  textures  are  displayed  for  comparison  with  the 
original  texture  image. 

The  texture  analysis  techniques  presented  in  Chapters 
2,  3,  and  4  can  be  used  for  more  than  texture  description 
generation.  In  Chapter  5  two  texture  analysis 
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applications  are  discussed.  A  scheme  for  texture 
recognition  (for  11  texture  classes)  is  presented.  Both 
texture  and  primitive  descriptions  are  used  in  the 
decision  process.  Results  for  11  natural  textures  are 
examined.  A  second  application  is  the  determination  of 
surface  orientation  from  texture.  A  surface  orientation 
analysis  algorithm  is  outlined.  It  uses  methods  suggested 
by  Bajcsy  [5]  and  Stevens  [23],  as  well  as  the  texture 
analysis  techniques  developed  in  this  thesis.  Preliminary 
results  for  a  number  of  texture  images  exhibiting  non-zero 
texture  gradients  are  discussed. 

In  the  final  chapter  a  summary  of  results  is  given 
along  with  a  discussion  of  the  possible  extensions  of  this 
work.  Areas  for  further  research  in  natural  texture 
analysis  are  discussed. 


CHAPTER  2 


AUTOMATIC  GENERATION  OF  DESCRIPTIONS  FOR 
NATURAL  TEXTURE  IMAGES 

2. 1  Introduction 

In  this  chapter  a  technique  is  presented  for 
automatically  generating  descriptions  of  natural  textures. 
These  descriptions  will  include  information  pertaining  to 
the  period  of  repetition  within  the  texture,  the  size  of 
textural  elements,  and  the  d irec' ional ity  of  the  texture. 
Although  this  information  will  not  completely  characterize 
the  texture  under  consideration  it  gives  a  strong 

indication  of  the  nature  of  the  texture  and  provides 

valuable  cues  for  further  analysis.  In  particular  the 
descriptions  generated  by  the  algorithm  presented  in  this 
chapter  will  be  used  as  the  starting  point  of  the 

primitive  extraction  algorithm  to  be  described  in 
Chapter  3. 

2.2  Some  Approaches  to  Texture  Feature  Extraction 

A  natural  statistical  approach  to  texture  analysis  is 
via  spatial  frequency  analysis.  A  two  dimensional  Fourier 
transform  of  an  image  would  be  expected  to  reveal 


periodicity  within  the  texture  field  as  well  as 
directional  information.  Bajcsy  [5]  developed  a  technique 
for  extracting  such  information  from  the  power  spectrum  of 
the  discrete  Fourier  transform,  i.e.,  from  the  magnitude 
of  its  complex  results.  By  studying  annular  sections  of 
the  power  spectrum,  directional  information  could  be 
detected.  A  flat  annular  function  would  indicate  a 
non-directional  texture,  while  peaks  in  this  section 
indicate  directionality.  In  the  case  of  a  non-directional 
texture,  peaks  in  power  spectrum  wedges  would  indicate  the 
presence  of  blob-like  structures  within  the  texture  field 
[6].  This  technique  has  been  successfully  used  on  images 
containing  various  types  of  natural  texture  regions. 

There  are  some  difficulties  in  working  with  the 
Fourier  transform  of  an  image.  In  natural  scenes,  where 
even  regular  or  periodic  textures  are  not  very  exact,  the 
power  spectrum  results  are  quite  diffuse.  Also,  the 
results  can  be  misleading  since  a  single  strong  edge 
within  the  image  may  have  the  effect  of  indicating 
textural  directionality.  An  example  of  a  set  of  natural 
textures  and  their  respective  2-D  Fourier  transform 
magnitude  is  given  in  Fig.  2-1. 

An  approach  aimed  at  defining  a  set  of  meaningful 
texture  features  to  be  extracted  from  natural  texture 
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(a)  Texture  Subwindows  (Aerial  (b)  Fourier  Transforms  of 
City,  Grass,  Sand,  Wool,  subwindows  in  (a) 

Raffia) 


(c)  Brodatz  Texture  Sub-  (d)  Fourier  Transfors  of 

windows  (Straw,  Water,  subwindows  in  (c) 

Wood,  Herringbone) 


Figure  2-1. 


Texture  Mosaics  and  their  Corresponding 
Fourier  Transforms  (from  [36]). 
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images  was  presented  by  Tamura,  Mori,  and  Yamawaki  in 
[25].  They  approximated  six  textural  features*  namely, 
coarseness,  contrast,  directionality,  line-likeness, 
regularity,  and  roughness,  and  compared  them  with 
psychological  measurement  taken  for  human  subjects.  They 
found  that  3  of  these  features  correlated  well  with  human 
subject  response.  These  were  the  coarseness,  contrast  and 
directionality  measures.  The  other  three  features  showed 
some  correspondence,  but  it  was  explained  that  further 
work  with  these  features  would  be  required.  A  possible 
problem  with  this  technique  is  the  subjectivity  of  human 
textural  interpretation.  A  different  group  of  subjects 
might  have  produced  completely  different  texture 
measurement  results. 

Due  to  the  Julesz  hypothesis  [1]  that  human  texture 
discrimination  depends  on  second-order  statistics,  much 
attention  has  been  given  to  the  spatial  gray  level 
dependence  matrix,  or  co-occurrence  matrix  technique 
proposed  by  Haralick,  Shanmugam,  and  Dinstein  [7]. 

The  co-occurrence  matrix,  P  (i,j),  is  an 

0 ,  d 

approximation  to  the  joint  probability  density  function 
defining  the  probability  of  a  pixel  with  intensity  level  i 
being  separated  by  angle,  6,  and  distance,  d,  from  a  pixel 
whose  intensity  level  is  j,  where  6  is  measured  with 
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respect  to  the  horizontal.  It  was  not  suggested  that  the 
matrix  elements  themselves  would  serve  as  textural 
features;  but  that  certain  measures  defined  on  them  would. 
For  example, 

V1  V1 

£ENT<6'd)  *  £  S  P6,d(i'3)l09(Pe,d(i'3))' 
i=0  j=0 

where  Ng  is  the  number  of  distinct  gray  levels  within  the 
quantized  image,  is  an  entropy  measure  on  the  elements  of 
P  .  (i,j).  In  [7]  twenty-eight  texture  features  are 

u  t  Cl 

suggested  for  use  in  texture  image  classification  schemes. 
(These  are  the  mean  and  range  for  14  co-occurrence 
measures.)  Image  class i f ica t io n  results  are  discussed  for 
3  separate  groups  of  natural  terrain  and  sand-stone 
images.  Different  sets  of  features  were  used  to  classify 
images  from  the  3  groups  to  produce  results  ranging 
between  82.3  and  89  percent  correct  classification. 

Further  endorsement  of  the  spatial  gray  level 
dependence  method  (SGLDM)  came  from  two  separate  studies. 
An  experimental  study  [26]  by  Weszka,  Dyer,  and  Rosenfeld 
and  a  theoretical  comparison  [27]  by  Conners  and  Harlow. 
In  both  studies  the  SGLDM  was  compared  to  3  other  methods 
the  gray  level  difference  method,  the  gray  level  run 


length  method,  and  the  power  spectrum  method. 


In  each 


case,  the  SGLDM  was  determined  to  be  the  most  powerful 
algorithm  of  the  four. 


In  [11]  Conners  uses  the  inertia  measure  to  determine 
unit  cell  size  within  the  texture  image.  The  inertia 
measures  is  defined  as  follows. 


/ 


where  Ng  is  the  number  of  gray  levels  within  the  quantized 
image . 

In  [11]  it  is  explained  that  this  measure  reaches  a 
minimum  when  the  two  unit  patterns  match  best  in  the  mean 
square  sense,  that  is,  when  the  square  of  the  intensity 
differences  is  minimized.  Therefore,  analysis  of  plots  of 
inertia  measures  over  varying  distances  for  a  variety  of 
angles  should  produce  information  about  the  tile  size  in 
regular  texture  images. 

One  drawback  in  using  this  method  is  its 
computational  complexity.  For  each  angle  and  distance  to 
be  examined  there  is  an  Ng  x  Ng  matrix  to  be  calculated. 
It  is  not  uncommon  to  have  8  bits  per  pixel,  requiring 
each  matrix  to  have  256  x  256  entries.  (It  should  be 
noted  that  calculations  are  usually  carried  out  for  no 
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more  than  16  quantized  gray  levels.)  Also,  in  order  for 
meaningful  statistics  to  be  calculated  a  reasonably  large 
texture  region  should  be  used. 

In  [8]  Price  and  Nevatia  used  the  co-occur rence 
matrix  technique  not  on  the  image  itself  but  on  an  edge 
image  corresponding  to  the  original  texture  image.  In 
such  an  edge  image,  non-maximal  points  along  rows  or 
columns  are  set  to  zero.  Hence,  an  edge  is  represented  by 
a  single  line.  Since  the  image  being  analyzed  is  made  up 
of  essentially  binary  points,  noting  the  presence  or 
absence  of  an  edge,  co-occur rence  analysis  would  be  quite 
simple  computationally.  It  remained  to  determine  whether 
meaningful  texture  features  could  be  extracted  from  a 
texture  edge  image.  This  is  the  subject  of  the  following 
section  (2.3). 

2.3  Using  Edge  Repetition  Arrays  for  Structural 

Feature  Extraction 

An  ERA  is  like  a  co-occur r ence  matrix  on  the  edge  and 
direction  images  corresponding  to  a  gray  level  image. 
Basically,  a  match  occurs  when  there  are  two  edges  with 
the  proper  directions  and  relative  locations. 


Figure  1-1  (b)  is  the  edge  image  mosaic  corresponding 
to  the  brick  image  mosaic  in  Fig.  1-1  (a)  .  These  edge 


images  retain  the  basic  structural  definition  of  their 
cor respond ing  gray  level  images.  Bricks  and  mortar  strips 
are  clearly  outlined,  thus  providing  not  only  primitive 
size  and  shape  information,  but  primitive  arrangement 
information  as  well.  Another  example  of  a  gray  level 
image  with  its  corresponding  edge  image  is  the  raffia 
(woven  palm)  texture  sample  given  in  Fig.  2-2.  The 
primitives  comprising  this  natural  texture  are  not  as 
uniform  as  the  brick  primitives  of  the  image  in 
Fig.  1-1  (a) .  This  is  reflected  in  its  corresponding  edge 
image.  These  edge  images  (Figs.  1-1  (b)  and  2-2  (b)  )  have 
been  thinned  by  non-max imal  suppression  to  produce  edges 
composed  of  single  lines. 

All  of  the  edge  images  used  in  this  thesis  have  been 
produced  by  using  the  set  of  5  x  5  convolution  masks  [28] 
shown  in  Fig.  2-3.  Each  of  these  masks  or  operators  is 
applied  on  every  5x5  window  within  the  gray  level  image 
to  produce  a  magnitude  and  a  direction  (1  of  12).  The 
magnitude  chosen  for  a  window  is  the  highest  magnitude  in 
absolute  value  over  the  six  convolution  results.  The 
direction  is  determined  by  the  mask  producing  the  maximum 
result,  and  the  sign  of  this  result  as  well.  In 
Fig.  2-4  (a)  the  12  directions  are  shown.  In  each  case, 
the  higher  intensity  or  lighter  pixels  are  to  the  right  of 
the  vector  when  facing  in  the  vector's  direction,  see 


(a)  Raffia  (Woven  Palm)  Texture  Imige 


(b)  Non-Maximal  Suppressed  Edges  from  (a) 


Figure  2-2. 


Raffia  and  Edges. 


25 


w 


-100 

-100 

0 

100 

100 

-loo 

11 

100 

100 

100 

-100 

-100 

0 

100 

100 

•100 

-70 

•1 

100 

100 

-100 

-100 

0 

100 

100 

-100 

-100 

0 

100 

100 

-100 

-100 

0 

100 

100 

-100 

•100 

•01 

70 

100 

-100 

-100 

0 

100 

100 

-100 

•100 

•100 

-11 

100 

(•1 

0° 

(0) 

10° 

100 

100 

100 

100 

100 

100 

100 

100 

100 

100 

-32 

79 

100 

100 

100 

100 

100 

100 

100 

100 

-100 

-92 

0 

92 

100 

0 

0 

0 

0 

0 

-100 

-100 

-100 

-79 

32 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

<c> 

90° 

Id) 

00° 

100 

100 

100 

100 

100 

100 

100 

IOC 

11 

-100 

100 

100 

100 

79 

-32 

100 

100 

01 

-70 

-100 

100 

92 

0 

-92 

-100 

100 

100 

0 

-100 

-100 

32 

-79 

-100 

-100 

-100 

100 

70 

-01 

-100 

-100 

-100 

-100 

-100 

-100 

-100 

100 

-11 

-100 

-100 

-100 

<«) 

120 

o 

(() 

150 

Figure  2-3.  Covolution  Masks. 


(b)  Horizontal  Edges 


(c)  Vertical  Edges 


Figure  2-4.  Edge  Directions  as  Determined 
by  Edge  Operators  of  Fig.  2-3 


Figs.  2-4 (b)  and  (c) .  In  this  way  each  pixel  in  the 
original  gray  level  image  is  replaced  by  an  edge  magnitude 
and  direction.  After  the  edges  have  been  thinned  the  edge 
and  direction  information  is  ready  for  analysis. 

Consider  the  top  left  edge  image  in  Fig.  1-1  (b)  .  If 
one  examines  a  vertical  strip  of  pixels  from  the  center  of 
the  line  of  bricks  a  pattern  could  be  established.  Edges 
separate  short  (length  2)  strips  of  zeros  correspond ing  to 
the  mortar  interiors  from  long  (length  12)  strips  of  zeros 
representing  the  brick  interiors.  If  this  happens  often 
enough  in  the  image  then  it  can  be  said  that  2  types  of 
texture  primitives  have  been  found.  One  has  vertical 
dimension  2  and  the  other  has  vertical  dimension  12.  A 
period  of  repetition  can  also  be  established  from  this 
sequence  of  pixel  values.  It  is  the  number  of  pixels  from 
the  beginning  of  one  string  of  12  zeros  to  the  beginning 
of  the  next  similar  string.  Hence,  a  one-dimensional  scan 
through  the  edge  image  can  uncover  meaningful  structural 
information.  ERAS  are  computed  by  considering  edge 
placements  in  pixel  strips  extending  in  6  directions  from 
every  pixel  in  the  edge  image. 

It  might  seem  at  first  that  suppressed  edges  alone 
would  provide  enough  information  to  easily  extract  the 
interesting  textural  features.  However,  when  scanning  in 
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a  certain  direction,  e.g.  horizontal,  and  a  like 
direction  (horizontal)  edge  is  encountered,  there  will  be 
a  match  at  every  distance  smaller  than  or  equal  to  the 
length  of  the  edge.  While  edge  lengths  may  be  a 
meaningful  measure  it  is  not  necessarily  helpful  in 
determining  period  and  element  siz^  within  the  image.  It 
was  therefore  decided  that  the  edge  directions  would  be 
used  to  determine  which  kinds  of  edge  matches  to  consider. 


In  particular,  only  edges  whose  directions  are 
perpendicular  to  the  direction  of  scan  are  considered 
potential  matching  edges. 

Both  element  size  and  spacing  (or  period)  information 
for  a  given  direction  is  produced  by  each  directional 
scan.  This  information  is  in  the  form  of  peaks  at  element 
size  and  spacing  values  and  their  repetitions.  A  natural 
next  objective  is  to  separate  the  information  for  these 
features  so  that  the  correct  interpretation  of  these 
matrices  can  be  expedited. 

Some  simple  observations  can  be  made  by  studying 
Fig.  2-4  (b)  and  (c).  Element  spacing  or  period  is  the 
distance  between  like  oriented  edges  while  oppositely 
oriented  edges  form  element  boundaries.  Therefore,  the 
distances  at  which  like  oriented  edges  match  measures 
element  spacing  in  that  direction,  while  the  distances  at 
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which  oppositely  oriented  edges  match  measures  the  size  of 
texture  primitives  in  the  given  direction.  Further,  it 
can  be  determined  whether  the  sizes  and  spacings  refer  to 
dark  or  light  intensity  objects.  For  examples,  in 
Fig.  2-4  (c)  a  dark  object  which  has  vertical  edges  will  be 
bounded  on  the  left  by  a  down-pointing  edge  and  on  the 
right  by  an  up-pointing  edge;  and  the  opposite  can  be  said 
of  light  objects  with  vertical  edges.  Therefore,  the 
intensity  of  the  element  under  consideration  can  also  be 
discovered  by  using  edge  and  direction  information. 

2.3.1  Algorithm  Details 

Some  of  the  details  of  the  ERA  method  as  used  in  this 
thesis  follow.  The  inputs  to  this  algorithm  are  the  edge 
and  direction  images  corresponding  to  the  original  texture 
image,  as  well  as  a  range  of  distances  over  which  to  scan. 

Edge  matches  are  considered  only  for  pairs  of  edges 
whose  directions  are  perpendicular  to  the  direction  of 


scan 

.  Therefore,  the  se 

t  of 
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in  Fig. 

2-4 (a)  . 

If 

more 

accuracy  is  needed  the  number  of  edge  directions  (and  scan 
directions)  could  be  increased.  However,  along  with  these 
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HORIZONTAL  SCAN 


SCAN 


(a)  Edge  Repetition  Array  Scan  Directions 


Dark  Element 
Spacing 


Light  Element 


Element  Element 

Size  Size 


Light 

Element  Size 


(b)  Types  of  Matches  for  Horizontal  Scan 


Figure  2-5. 


ERA  Scan  Directions  and  Types  of  Horizontal 
Edge  Matches 


increases  there  would  be  a  proportional  increase  in  the 
amount  of  time  and  space  needed  for  the  execution  of  this 
algorithm. 


Separate  matrices 

are 

calculated  for 

spacings 

and 

sizes  of  both  dark 

and 

light  intensity 

elements. 

For 

example,  when  scanning 

horizontally  the 

following 

four 

types  of  matches  are  sought  at  each  distance  within  the 
user  specified  range: 

1.  Matching  pairs  of  vertical  down-pointing  edges,  which 
indicate  element  spacing  for  dark  objects? 

2.  A  down-pointing  vertical  edge  followed  by  an 
up-pointing  vertical  edge,  which  indicate  element  size 
for  dark  objects; 

3.  Matching  pairs  of  vertical  up-pointing  edges,  which 
indicate  element  spacing  for  light  objects;  and, 

4.  An  up-pointing  vertical  edge  followed  by  a 
down-pointing  vertical  edge,  which  indicate  element 
size  for  light  objects.  (See  Fig.  2—5  ( b)  )  . 

Information  for  twenty-four  separate  graphs  is 
generated  for  each  texture  image.  However,  there  are  only 
4  values  for  each  (angle,  distance)  pair.  Each  ERA  entry 
is  the  number  of  matches  of  a  particular  type,  for  example 
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two  like  up-pointing  vertical  edges,  occurring  at  a  given 
angle  and  distance.  This  number  is  then  normalized  by  the 
number  of  matches  added  to  the  number  of  times  only  one  of 
the  two  desired  edges  occurred  in  the  proper  location.  It 
is  then  multiplied  by  100.0  to  form  a  percentage.  It 
tells  what  percent  of  the  time  a  match  occurred  given  that 
at  least  one  of  the  edges  appeared  in  the  proper  place. 

The  entries  are  not  normalized  by  the  number  of 
possible  locations  within  the  image  in  which  the  given 
angle  and  distance  relationship  is  satisfied  because  this 
would  cause  the  values  of  the  entries  to  depend  on  the 
size  of  the  objects  or  periods  being  measured.  In 
Fig.  2-6,  the  effects  of  these  two  normalization  schemes 
are  illustrated.  NORM  1  is  the  total  number  of  locations 
where  a  match  can  occur.  NORM 2  is  the  number  of  locations 
where  at  least  one  of  the  two  matching  edges  occurs  in  the 
correct  location.  In  each  case  all  of  the  edge  points 
were  present  for  the  given  period  size.  However,  NORM 1 
causes  the  entry  value  to  decrease  as  the  period  size 
increases  while  N0RM2  allows  the  entry  value  to  remain 
constant.  Each  ERA  entry  is  of  the  form: 

100  Number  of  matches  at  distance  d 
Normalization  count  for  d 

where  d  is  within  the  user  specified  range. 
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Figure  2-6.  Comparison  of  2  Normalization  Schemes. 

(Actual  ERA  Entries  are  then  Multiplied 
by  100.0.) 
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The  order  of  this  algorithm  is  DP,  where  D  is  the 
number  of  distances  to  be  examined  at  each  angle  and  P,  is 
the  number  of  pixels  in  the  image.  The  average  CP'J  time 
needed  to  calculate  a  set  of  ERAs  for  a  128  x  128  pixel 
image  at  31  distances  is  1.5  minutes  (on  a  DEC  KL-10 
processor).  For  purposes  of  comparison  it  is  noted  that 
six  5x5  masks  can  be  convolved  with  a  128  x  128  pixel 
image  to  produce  edge  magnitudes  and  directions  for  each 
pixel  in  15  CPU  seconds. 

2.3.2  Some  ERA  Results 

The  edge  repetition  arrays  for  the  raffia  texture 
image  in  Fig.  2-2  (a)  are  given  in  Fig.  2-7.  The  element 
spacing  graphs  are  on  the  left,  the  element  size  graphs 
are  on  the  right.  The  broken  line  graphs  are  light 
element  ERAs,  while  the  solid  line  graphs  are  for  dark 
elements.  The  two  numbers  appearing  below  each  graph  are 
the  normalization  numbers  for  the  smallest  distance 
entries  for  dark  and  light  elements,  respectively.  These 
numbers  give  a  rough  idea  of  the  number  of  edges  found  in 
that  particular  scan  direction. 

First,  consider  the  actual  texture  image 
(Fig.  2-2  (a)).  Raffia  is  a  natural  texture  which  is 
periodic  in  the  vertical  direction  due  to  the  similarity 
of  widths  of  the  pieces  of  palm  used  in  the  weave.  It  is 


only  loosely  periodic  in  the  horizontal  direction  due  to 
the  irregular ities  of  the  weave.  The  predominant  texture 
primitives  are  bright  rectangular  regions  and  the  dark 
elongated  regions  separating  them.  In  some  parts  of  the 
image  a  light  vertical  strip  (highlight)  separates  the 
large  rectangles.  This  can  be  considered  a  third 
primitive  type. 

Next,  consider  the  ERA  results  shown  in  Fig.  2-7. 
The  most  significant  results  appear  in  the  vertical  and 
horizontal  scan  directions.  The  vertical  element  spacing 
graphs  Cor  both  dark  and  light  intensity  objects  have 
peaks  at  multiples  of  8,  which  indicates  a  vertical  period 
of  3  pixels.  Dark  objects  with  vertical  dimension  3  are 
strongly  indicated  by  the  high  amplitude  peak  at  pixel 
distance  3  in  the  solid  line  element  size  graph  for  the 
vertical  scan  direction.  Likewise,  a  vertical  element 
size  of  5  is  strongly  indicated  for  light  objects.  It 
should  be  noted  that  peaks  also  occur  at  intervals  of 
approximately  8  starting  from  the  initial  element  size 
peaks  for  both  light  and  dark  objects.  This  is  further 
indication  of  the  vertical  period  of  8  pixels. 

The  most  significant  peak  in  the  set  of  horizontal 
graphs  occurs  in  the  light  element  size  graph  at  pixel 
distance  3.  No  obvious  peak  repetitions  occur. 
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The  information  one  can  extract  from  these  graphs 


agrees  well  with  the  structural  description  of  the  image, 
when  considering  one  dimension  at  a  time. 

Figure  2-8  (a)  is  a  texture  image  mosaic  containing 
samples  of  4  regular  textures.  Each  subimage  is  128  x  128 
pixels.  The  corresponding  edge  image  mosaic  is  given  in 
Fig.  2-8 (b).  There  are  2  aerial  images  of  a  section  of 
San  Francisco  in  the  top  row  of  the  mosaic.  In  the  second 
row  there  is  a  picture  of  a  floor  grating  on  the  left,  and 
a  piece  of  herringbone  material  on  the  right.  Figures  2-9 
through  2-12  are  the  edge  repetition  arrays  for  these 
texture  images. 

In  the  leftmost  city  image  the  streets  line  up 
exactly  with  the  vertical  and  horizontal  scan  directions. 
Therefore,  one  would  expect  the  strongest  ERA  results 
appear  in  the  vertical  and  horizontal  graphs.  This  is 
shown  in  Fig.  2-9.  In  the  horizontal  element  spacing 
graphs  for  both  dark  and  light  elements  there  is  strong 
evidence  of  a  period  size  of  7  pixels.  This  is  supported 
in  the  horizontal  element  size  graphs  since  peaks  in  these 
graphs  repeat  at  intervals  of  7.  The  first  element  size 
peaks  are  at  3  for  dark  objects  and  4  for  light  objects. 
These  sizes  corresponds  to  the  dark  vertical  strips  and 
the  light  vertical  strips  separating  them,  respectively. 


\ 


39 


(a)  Texture  Mosaic  Including  Cityl,  City2, 
Herringbone,  and  Grating  Images 


(b)  Non-Maximal  Suppressed  Edges  From  (a) 
Figure  2-8.  Periodic  Texture  Mosaic  and  Edges 
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The  vertical  scan  direction  ERAs  also  exhibit  signs  of 


periodicity.  The  period  for  both  dark  and  light  elements 
is  16  in  the  vertical  direction.  The  element  sizes  in 
this  direction  are  3  for  light  objects  and  13  for  dark 
objects.  It  would  seem  that  the  same  dark  strips  were 
found  in  a  second  scan  direction  and  that  the  shorter 
light  regions  above  and  below  these  dark  areas  are  the 
light  primitives  with  vertical  dimension  3. 

The  second  aerial  city  picture  is  a  picture  of  the 
same  general  area  taken  at  a  different  angle  and 
resolution.  As  expected  the  strongest  results  are  in  the 
30  degree  scan  direction  which  is  the  closest  scan  angle 
perpendicular  to  the  street  direction,  see  Fig.  2-10(a). 
The  30  degree  scan  picks  up  2  distinct  element  sizes  per 
intensicy  repeating  at  the  same  periods.  Looking  at  the 
image,  this  would  seem  to  be  caused  by  the  fact  that  there 
are  3  types  of:  colored  strips  in  each  line  of  city  blocks, 
a  black  strip  beside  a  dark  gray  strip,  beside  a  light 
gray  strip.  In  Fig.  2-10 (b)  the  effect  of  this  phenomenon 
is  examined.  As  one  would  expect,  one  light  element 
period  (14),  and  3  dark  element  periods  (6,  9,  and  15)  are 
found.  Also  two  light  element  sizes  (5  and  10)  and  two 
dark  element  sizes  (5  and  10)  can  be  found  in  this  same 
scan  direction.  Similar  sizes  and  spacings  can  be  found 
in  the  horizontal  direction.  Due  to  slight  irregular ities 
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Figure  2-10 (a).  City2  ERA  Graphs 
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Element  Periods 


in  the  shapes  of  the  primitives  some  edges  were  determined 


to  be  90  degrees  instead  of  120  degrees.  This  would 
explain  returns  in  the  horizontal  scan  direction.  Element 
size  peaks  also  occur  at  other  scan  directions  but  no 
periodic  repetitions  seem  to  occur. 

Herringbone  material  is  the  texture  occurring  in  the 
lower  left  section  of  the  texture  mosaic  of  Fig.  2-8  (a) . 
At  first  glance  one  might  be  tempted  to  think  that  most  of 
the  elements  had  orientation  45  or  135  degrees  and  so 
element  sizes  and  periods  would  be  least  likely  to  appear 
in  the  vertical  and  horizontal  scan  directions.  However, 
a  closer  look  ;  this  texture  reveals  that  the  textural 
primitives  are  really  little  rectangular  regions  chained 
together.  These  rectangles,  which  are  actually  the 
threads  of  which  the  material  is  constructed,  have  most  of 
their  edges  occurring  with  direction  0  or  90  degrees.  The 
ERAs  with  the  strongest  returns  should  then  be  in  the 
vertical  and  horizontal  directions. 

In  Fig.  2-11 (a),  we  see  that  both  the  vertical  and 
horizontal  element  spacing  graphs  report  strong  evidence 
of  periodicity.  The  periodicity  is  the  same  for  both  dark 
and  light  intensity  objects.  Multiple  element  sizes  are 
apparent  for  both  dark  and  light  objects.  This  can  be 
explained  by  the  weave  of  the  material.  (See 
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Dork  Horizontal  Element  Size 


Figure  2-11  (b).  Herringbone  Weave  Element  Sizes 


Fig.  2—1 1 ( b) ) .  In  all  of  the  remaining  scan  directions 
some  element  sizes  can  be  found  but  there  is  no  strong 
evidence  of  repetition. 

The  final  image  in  this  mosaic  is  the  floor  grating 
(black  dot)  pattern.  The  ERAs  in  Fig.  2-12  show  that 
every  scan  direction  produces  element  size  and  spacing 
peaks  for  both  dark  and  light  intensity  objects.  This  is 
not  surprising  since  the  boundary  of  the  black  dots  must 
approximate  a  circle  and  so  each  possible  edge  direction 
should  be  represented.  The  smallest  dark  element  size 
peak  is  approximately  8  for  all  directions,  while  the 
smallest  light  element  size  peaks  vary  widely.  This  can 
be  explained  by  the  black  dot  diameters  being  roughly 
constant  while  the  size  of  the  light  region  between  the 
dots  varies  with  the  scan  direction. 

In  Fig.  2-13(a),  there  is  another  natural  texture 
mosaic.  The  4  texture  subwindows  are  each  128  x  128 
pixels.  None  of  the  textures  in  this  image  are  regular, 
i.e.,  periodic.  However,  in  some  it  seems  that  a  pattern 
does  emerge.  The  top  left  texture  is  a  picture  of  ocean 
waves;  to  the  right  of  it  is  a  wood  grain  texture.  In  the 
second  row  there  are  2  pictures  of  straw,  one  which  is 
uni-directional,  and  one  which  has  pieces  of  straw  at 
several  different  orientations.  Figure  2-13(b)  is  a 
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Figure 


a)  Texture  Mosaic  Including  Water, 
Wood,  Strawl,  and  Straw2  Images 


(b)  Non-Maxima 1  Suppressed  Edges  from  (a) 
2-13.  Non-Periodic  Texture  Mosaic  and  Edges 
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mosaic  of  the  edge  images  corresponding  to  the  texture 
images  in  (a) . 

An  observation  one  might  make  concerning  the  two  top 
row  images  in  Fig.  2-13  (a)  is  that  they  are  somewhat 
similar.  The  primitives  of  both  are  elongated  or 
line-like  dark  regions,  oriented  in  a  vertical  direction. 
One  distinguishing  feature  is  the  length  of  these  objects 
(shorter  for  the  water  image)  .  Looking  at  the  ERAs  for 
the  water  (Fig.  2-14)  and  wood  grain  (Fig.  2-15)  images 
one  can  observe  their  similarity  as  well.  The  only  ERA 
graphs  showing  significant  results  are  the  ones  in  the 
horizontal  scan  direction.  No  period  seems  to  be  evident 
for  either  water  or  wood.  However,  there  are  strong 
indications  for  element  size  of  3  for  dark  and  light 
elements  in  both  texture  images.  Since  there  are  many 
small  elements  with  vertical  edges  randomly  spaced 
throughout  the  image,  many  element  spacing  matches  and 
element  size  repetitions  will  occur  in  the  horizontal  scan 
direction  at  all  distances  giving  the  graph  a  high  floor 
value  after  the  initial  peak  and/or  valley.  This  type  of 
graph  should  be  characteristic  of  highly  directional 
random  patterns,  i.e.,  patterns  with  a  high  density  of 
elements  oriented  in  the  same  direction. 

In  the  bottom  row  of  the  texture  mosaic 
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(a) 


Texture  Mosaic  Includiny  Grass, 
Sand,  Wool  1  and  Wool  2  Images 


(b)  Non-Maximal  Suppressed  Edges  from  (a) 
Figure  2-18.  Random  Texture  Mosaic  and  Edges 
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subimages.  The  textures  included  in  this  mosaic  are  the 
most  random  ones  to  be  examined.  The  edge  image 
corresponding  to  the  mosaic  in  Fig.  2-18 (a)  is  given  in 
Fig.  2-18  (b).  The  textures  represented  in  these  mosaics 
are  grass  and  sand  in  the  first  row  from  left  to  right, 
and  two  wool  images  in  the  second  row. 

The  ERAs  for  grass  and  sand  are  given  in  Figs.  2-19 
and  2-20,  respectively.  Neither  pattern  shows  any 
evidence  of  periodicity  but  both  show  evidence  of  element 
size  in  their  horizontal  and  vertical  scan  direction 
graphs.  The  predominant  element  size  for  grass  is  3  for 
both  dark  and  light  objects.  Element  sizes  for  sand  are  3 
for  dark  objects  and  4  for  light  objects. 

The  information  extracted  for  these  two  textures  is 
very  similar,  even  though  the  textures  look  quite 
different  to  the  human  observer.  After  taking  a  cursory 
look  at  the  sand  texture  image  one  would  be  tempted  to  say 
that  the  sand  texture  was  made  up  of  perfectly  circular 
pieces  of  sand,  all  extremely  close  in  size,  but  scattered 
randomly  throughout  the  ^image.  However,  on  closer 
inspection  one  will  notice  that  only  relatively  few  of  the 
sand  pebbles  in  the  image  are  round  and  there  is  great 
variation  with  respect  to  pebble  size.  The  shadows 
between  the  pebbles  are  probably  more  of  a  constant  within 
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the  image.  It  would  seem  then  that  the  sand  element  size 
peaks  are  measuring  only  the  smallest  of  the  sand  pebbles 
and  the  spaces  between  grains  and  not  the  large  circular 
pebbles  which  our  eye  tends  to  pick  out. 

The  two  texture  images  in  the  second  row  of  the 
mosaic  in  Fig.  2-18 (a)  are  wool  texture  subwindows.  Their 
ERAS  appear  in  Figs.  2-21  and  2-22.  As  in  grass  and  sand 
the  strongest  amplitudes  are  in  the  horizontal  and 
vertical  scan  direction  element  size  graphs.  The  results 
for  dark  elements  are  stronger  than  those  for  light 
elements  in  both  images.  There  is  very  weak  evidence  for 
periodicity  of  dark  elements  in  the  horizontal  scan 
direction.  When  we  look  at  the  wool  images  in 
Fig.  2-18(a),  we  have  no  trouble  finding  squares  which 
repeat  at  definite  periods.  A  closer  look  will  reveal 
that  in  fact  there  are  very  few  if  any  square  elements  in 
the  image.  The  one  dimensional  scans  have  a  better  chance 
of  finding  the  dark  strips  (shadows)  which  separate  some 
of  the  "square"  sections. 

2.4  Automatic  Interpretation  of  ERA  Results 


The  results  given  in  Section  2.3.2 

have 

shown 

that 

the  edge 

repetition  array 

is  a 

use  f  ul 

dev  ice 

for 
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properties. 
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next  step  in  the  texture  analysis  process  is  to  automate 
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the  ERA  interpretation  process.  That  is,  to  automatically 
generate  texture  descriptions  from  ERA  results. 

Before  one  can  design  an  algorithm  to  perform  an 
intelligent  process  it  is  necessary  to  have  a  clear 
understanding  of  the  process  one  wishes  to  imitate.  The 
following  is  a  brief  descriptive  outline  of  the  approach 
taken  by  the  author  to  extract  the  pertinent  structural 
information  from  sets  of  ERA  graphs. 

2.4.1  The  ERA  Interpretation  Process 

For  each  (direction,  intensity)  pair  there  are  two 
graphs,  namely,  an  element  spacing  graph  and  an  element 
size  graph.  The  first  step  in  the  examination  process  is 
to  locate  and  classify  all  of  the  peaks  indicated  in  this 
pair  of  graphs.  When  the  most  significant  or  highest 
peaks  are  found  it  is  necessary  to  determine  whether  they 
carry  important  structural  information  or  are  simply  part 
of  the  noise.  If  both  graphs  are  nearly  constant  all  that 
can  be  said  is  that  there  are  no  obvious  element  sizes  or 

■  i 
\> 

periods.  However,  when  peaks  with  significant  peak  to 
valley  separation  are  found  the  next  level  of  analysis  can 
beg  in . 

If  the  element  spacing  graph  shows  strong  peaks  at 
all  multiples  of  a  certain  distance  then  periodicity  in 


the  scan  direction  of  the  graph  pair  is  suspected.  When 
this  is  further  supported  by  element  size  peaks  repeating 
at  intervals  of  the  indicated  spacing  value  then  this 
hypothesis  is  confirmed.  The  texture  is  considered  to  be 
regular  and  the  period  and  directionality  are  known.  At 
this  point  it  is  necessary  to  determine  the  sizes  of  the 
textural  elements  or  primitives  in  the  given  scan 
direction. 

If  there  are  no  indications  of  periodicity,  but  there 
is  at  least  one  strong  peak  in  the  element  size  graph  then 
it  is  assumed  that  the  texture  is  composed  of  randomly 
placed,  similar-sized  textural  elements.  The  dimension  of 
the  texture  primitives  in  the  indicated  scan  direction  is 
taken  to  be  the  pixel  distance  at  which  the  peaks  in 
element  size  values  occur.  If  there  is  a  high  floor  value 
following  the  initial  element  size  peak  then  it  is  likely 
that  there  is  a  high  density  of  edges  oriented 
perpendicular  to  the  direction  of  scan.  Since  there  are 
many  matches  at  every  element  size,  their  placement  is 
probably  random.  In  this  case  the  texture  under 
consideration  is  determined  to  be  a  highly  d i rect ional 
random  pattern. 

The  analysis  described  above  is  repeated  for  each 
(direction,  intensity)  pair,  producing  a  structural 
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description  for  the  texture  with  respect  to  light  and  dark 
intensity  objects  in  each  of  six  scan  directions. 

2.4.2  Descriptive  Outline  of  the  ERA 
Interpretation  Program 

The  program  designed  to  automatically  interpret  ERA 
results  follows  the  procedure  described  in  Section  2.4.1 
very  closely.  The  first  program  section  deals  with  peak 
detection  and  classification.  Initially  each  point  in  the 
graph  is  labelled  as  a  peak,  a  valley,  or  a  hillside.  All 
peaks  are  catalogued  with  certain  descriptive  information, 
e.g.,  peak  width,  amplitude,  valley  to  valley  width,  etc. 
An  attempt  is  then  made  to  classify  the  peaks  as  major, 
minor  or  subminor.  This  classification  process  takes  into 
account  both  absolute  and  relative  peak  amplitudes.  A 
detailed  description  of  the  classification  process  is 
given  in  Section  2.4.3.  Even  the  largest  peaks  in  a  graph 
will  be  ignored  if  they  are  not  above  a  reasonable  value, 
as  in  the  150  degree  scan  direction  graphs  in  Fig.  2-14. 
On  the  other  hand,  all  of  the  peaks  in  a  graph  might  be 
classified  as  major.  See  the  horizontal  scan  direction 
graphs  in  Fig.  2-9.  Classification  of  the  peaks  in  these 
2  examples  is  fairly  straightforward.  However,  there  are 
cases  where  the  classification  decision  is  not  as  clear 
cut . 
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An  attempt  has  been  made  to  design  an  algorithm 
flexible  enough  to  correctly  identify  meaningful  graphic 
information  even  when  the  amplitudes  are  fairly  low. 
However,  some  absolute  and  dynamic  cutoffs  are  used  in  an 
attempt  to  filter  out  misleading  or  noisy  information. 
For  example,  no  peak  with  amplitude  less  than  2  is 
considered  as  a  major  or  minor  peak.  This  is  reasonable 
when  we  consider  that  the  value  2  means  that  a  match 
occurred  only  2  percent  of  the  time  that  at  least  one  of 
the  edge  types  necessary  for  the  match  was  in  the  proper 
location.  Clearly,  a  peak  with  amplitude  2  does  not 
reflect  any  major  structural  feature  of  the  texture 
pattern . 

Choosing  the  value,  2,  as  a  cutoff  does  not  mean  that 
peaks  with  amplitude  2.0  are  considered  to  be  far  more 
important  than  peaks  with  amplitude  1.9.  The  threshold 
2.0  is  a  conservative  choice.  It  is  low  enough  so  that  no 
significant  textural  feature  is  in  danger  of  being  missed. 
However,  it  is  useful  in  that  many  small  peaks  which  are 
part  of  the  noise  are  excluded  from  further  consideration. 

Along  with  peak  detection  and  classi f ication  some 
general  graph  descriptors  are  calculated  in  this  program 
section.  One  description  classifies  each  graph  as  to  how 
closely  it  approximates  a  constant.  Another  descriptor 
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flags  graphs  which  have  a  high  floor  value  after  their 
initial  peak  and  valley.  Therefore,  graphs  which  have  a 
high  floor  value  after  an  initial  high  peak,  an  initial 
low  valley,  or  a  high  peak  followed  by  a  low  valley  are 
flagged . 

The  next  major  program  section  determines  whether  the 
texture  is  periodic  in  the  given  scan  direction,  and  if  so 
it  determines  its  period.  Major  peaks  in  the  element 
spacing  graph  are  examined  to  determine  if  peaks  exist  at 
all  multiples  of  a  given  spacing  value.  If  this  occurs 
then  all  but  the  first  peak  are  tagged  as  multiples  and 
the  next  untagged  major  element  spacing  peak  is  examined. 
In  this  process  the  strength  of  the  repetition  is  noted. 
If  there  are  major  peaks  at  all  the  multiples  of  a  given 
value  the  repetition  is  considered  to  be  stronger  than  if 
only  minor  peaks  occur  at  the  multiples  of  this  value. 
The  repetition  descriptor  assigned  has  one  of  4  possible 
values  {FAIL,  IC,  PK,  or  MPK) .  The  exact  criterion  used 
to  assign  these  values  is  explained  in  detail  in  a  later 
section. 

After  all  of  the  major  element  spacing  peaks  have 
been  considered  as  possible  periods  of  repetition  the 
element  size  graph  is  analyzed.  Support  for  potential 
element  spacing  values  is  sought.  Each  major  and  minor 


element  size  peak  is  examined  to  determine  if  repetitions 
occur  at  intervals  of  a  potential  element  spacing  value. 
That  is,  if  M  is  a  possible  texture  period  value  and  N  is 
a  value  at  which  a  major  or  minor  element  size  peak 
occurs,  then  peaks  are  sought  at  the  ele.-.ent  size  values 
of  N+m,  N+2M,  N+3M,  etc.  If  such  peaks  are  found  then  the 
strength  of  repetition  is  noted  and  all  repetitions  of  the 
element  size  peak  are  tagged  with  respect  to  the  specific 
element  spacing  peak  under  consideration. 

The  final  section  of  this  program  is  the 
interpretation  or  description  generation  section.  All  of 
the  descriptors  previously  generated  are  considered  in  a 
specific  order.  This  program  section  is  logically 
equivalent  to  a  small  production  system  with  rules  of  the 
form: 

IF (CONDI  AND  C0ND2  AND... AND  CONDN ) 

THEN  PRINT (Appropr iate  Descriptive  Message) 

Some  specific  examples  of  the  production  rules  used  are 
given  at  the  end  of  the  following  section. 

2.4.3  The  ERA  Interpretation  Program  in  Detail 

In  this  section  the  ERA  interprets tion  program  will 
be  explained  in  detail.  The  major  program  sections  to  be 
discussed  below  perform  the  following  tasks: 


1)  Peak-Valley  Signature  Calculation 


2)  Amplitude  Descriptor  Calculation 

3)  Constant  Profile  and  Directionality  Descriptor 

Calculation 

4)  Peak  Information  Collection 

5)  Major  Peak  Extraction 

6)  Repetition  Descriptor  Calculation 

7)  Description  Generation 

Peak-Valley  S ignature  Calculation 

A  peak-valley  signature  (P-VS)  is  calculated  for 
every  element  size  and  spacing  ERA.  This  is  a  vector  of 
dimension  D  =  M2-M1+1,  where  Ml  and  M2  are  the  smallest 
and  largest  distances  to  be  scanned  when  calculating  the 
ERAs.  Each  P-VS  value  is  either  a  P,  V,  or  H  depending  on 
whether  the  corresponding  ERA  value  is  a  peak,  a  valley, 
or  a  hillside,  respectively.  Plateaus  occur  when  2 
neighboring  values  differ  by  less  than  the  minimum  of  one 
third  of  the  amplitude  of  the  ERA  entry  being  considered 
and  the  value  .6.  As  an  example,  consider  the  solid  line, 
vertical  scan  direction,  element  spacing  graph  given  in 
Fig.  2-12.  The  P-VS  values  for  the  distances  18  through 
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32  are  as  follows:  V,  V,  H,  H,  H,  H,  P,  H,  H,  H,  H,  H,  V, 
P,  V.  Later  in  the  program,  when  peaks  are  further 
classified  as  major,  minor,  or  subminor  the  peaks  in  the 
signature  are  adjusted  to  reflect  this  classification. 
The  P-VS  is  used  to  provide  easy  access  to  location,  width 
and  classification  of  peaks. 

Ampl itude  Descriptor  Calculation 

The  amplitude,  A,  of  each  point  on  the  graph  is 
classified  as  follows: 

a)  VERY  STRONG  if  A  is  greater  than  15; 

b)  STRONG  if  A  is  greater  but  not  greater  than 

15; 

c)  MODERATE  if  A  is  greater  than  5  but  not  greater  than 
10; 

d)  WEAK  if  A  is  greater  than  2  but  not  greater  than  5; 
and , 

e)  VERY  WEAK  if  A  is  not  greater  than  2. 

Constant  Profile  and  Directionality 
Descr iptor  Calculation 

Both  of  these  calculations  are  performed  in  order  to 
provide  information  about  the  overall  shape  of  the  graph. 
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The  following  3  conditions  are  described: 

1)  near  constant  amplitude  throughout  the  ERA; 

2)  relatively  constant  amplitude  starting  with  the  second 
ERA  peak;  and, 

3)  high  ERA  floor  value  over  all  but  a  small  number  of 
points. 

A  near  constant  graph  has  little  structural 
information  to  offer.  It  indicates  that  no  predominant 
element  size  or  spacing  is  likely  to  be  found  in  the  given 
scan  direction.  An  example  is  the  30  degree  scan 
direction  ERA  in  Fig.  2-15. 

Some  textures  have  like-sized  elements,  but  they  are 
randomly  placed.  In  this  case  a  predominant  element  size 
will  be  indicated  by  a  high  amplitude  peak  in  the  element 
size  graph.  However,  since  the  texture  is  not  periodic, 
element  size  repetitions  at  every  distance  may  be  equally 
likely.  Therefore,  after  the  first  peak,  the  graph  may 
tend  to  be  relatively  constant.  An  example  is  the 
horizontal  ERA  for  light  objects  in  Fig.  2-17. 

A  graph  with  most  of  its  points  above  a  relatively 
high  value  usually  indicates  that  there  is  a  high  density 
of  edges  with  direction  perpendicular  to  the  scan 
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direction.  If  the  texture  were  periodic  then  deep  valleys 
should  be  found  between  multiples  of  the  element  spacing 
distance,  therefore  it  is  also  likely  that  the  texture  is 
random.  A  texture  with  a  graph  exhibiting  a  high  minimum 
value  over  most  of  its  entries  will  then  be  labelled  as  a 
directional  random  pattern.  The  horizontal  ERAS  in 
Fig.  2-14  exhibit  this  phenomenon. 

Descriptors  for  these  3  general  classes  of  graphs  are 
calculated  as  follows: 

1)  If  the  difference  between  the  highest  and  lowest  graph 
values  is  not  greater  than  two(three)  the  graph  is 
considered  to  be  strongly  (moderately)  constant. 

2)  If  the  different  between  the  highest  and  lowest  graph 
values  beginning  with  the  second  peak  is  less  than  the 
maximum  of  one-fourth  of  the  largest  graph  value  and 
the  value  2  then  the  ERA  is  considered  to  be  strongly 
constant  after  the  first  peak.  If  this  difference  is 
less  than  the  maximum  of  one-third  of  the  maximum 
graph  value  and  the  value  3  then  the  ERA  is  considered 
to  be  moderately  constant  after  the  first  peak. 

3)  Let  T  ■  M2-M1-2,  i.e.,  all  but  3  points  of  the  graph, 
where  Ml  and  M2  are  the  smallest  and  largest  pixel 
distances  to  be  considered  when  calculating  the  ERA. 
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If  at  least  T  graph  values  are  greater  than  or  equal 
to  the  maximum  of  one-fourth  of  the  largest  graph 
amplitude  and  the  value  3  then  it  is  assumed  that  the 
texture  is  a  highly  directional  random  pattern.  If  at 
least  T  graph  values  are  greater  than  or  equal  to  the 
maximum  of  one-fifth  of  the  largest  graph  amplitude 
and  the  value  2  then  it  is  assumed  that  the  texture  is 
a  moderately  directional  random  pattern.  If  neither 
occurs  then  the  directionality  descriptor  is  set  to 
zero  to  indicate  that  the  texture  is  non-directional . 

If  neither  (1)  nor  (2)  above  occurs  then  the  constant 
profile  is  set  to  zero.  If,  at  a  later  point  in  the 
program,  it  is  obvious  that  the  texture  is  periodic  in  the 
given  direction  and  the  directionality  descriptor  is  zero 
for  both  element  spacing  and  size  graphs  then  the  constant 
profile  for  these  2  graphs  set  to  zero.  In  short, 
evidence  of  periodicity  is  given  preference  over  evidence 
of  randomness  in  borderline  cases. 

The  thresholds  used  in  this  section  were  chosen  after 
examining  ERA  graphs  for  a  wide  range  of  textures.  Both 
dynamic  and  absolute  thresholds  are  used  to  lend 
flexibility  to  the  basic  classification  scheme.  These 
thresholds  were  conservatively  chosen  to  insure  that  their 
use  would  not  affect  the  robustness  of  the  algorithm. 
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Peak  Information  Collection 


The  following  information  is  catalogued  for  each  peak 
for  use  later  in  the  program: 

a)  peak  amplitude; 

b)  peak  width; 

c)  valley  to  valley  width; 

d)  location  of  left  valley; 

e)  location  of  right  valley; 

f)  difference  in  amplitude  between  peak  and  left 
valley;  and, 

g)  difference  in  amplitude  between  peak  and  right 
valley. 

A  list  of  peaks  is  also  formed  in  this  section  to 
facilitate  the  sequential  examination  of  peaks.  Peaks 
with  maximum  peak-valley  variation  (the  maximum  of  c  and  d 
above)  greater  than  a  minimum  of  2  are  labelled  minor 
peaks.  At  a  later  point  in  the  program,  some  of  the  minor 
peaks  may  be  reclassified  as  major  peaks.  At  this  point, 
if  a  peak  is  not  labelled  as  minor  its  classification  will 
remain  as  subminor. 


Major  Peak  Extraction 


The  list  of  major  peaks  is  formed  in  this  program 
section.  There  are  two  separate  sets  of  criteria  for 
major  peak  classification.  The  first  set  pertains  to  both 
element  spacing  and  element  size  peaks.  However,  the 
second  set  is  used  only  for  element  spacing  peaks.  This 
latter  set  of  conditions  enables  an  element  spacing  peak 
which  would  normally  be  classified  as  minor  to  be 
considered  along  with  the  other  major  element  spacing 
peaks  in  the  search  for  texture  period.  It  is  hoped  that 
if  the  peak  representing  the  period  of  the  texture  falls 
short  of  the  basic  set  of  major  peak  conditions  then  the 
second  set  of  conditions  will  provide  enough  algorithm 
flexibility  to  make  up  for  it.  The  first  set  of  major 
peak  conditions  are  as  follows: 

a)  the  peak  amplitude  must  be  greater  than  or  equal  to 
the  maximum  of  two-thirds  of  the  maximum  graph  value 
and  3;  and, 

b)  both  left  and  right  peak  to  valley  differences  must  be 
greater  than  3. 

In  (a),  an  effort  is  made  to  select  the  highest 
element  size  and  spacing  peaks  in  each  graph.  However, 
none  below  amplitude  3  are  considered.  In  (b) ,  the  degree 
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of  peak  definition  is  examined.  Thus,  major  peaks  should 
be  among  the  highest  peaks  in  the  graph  and  have 
well-defined  peak  to  valley  separations.  Since  the 
texture  period  i^  such  an  important  piece  of  structural 
information  a  second  set  of  constraints  has  been  added  to 
allow  weaker  element  spacing  peaks  to  be  considered  along 
with  major  peaks  in  the  search  for  texture  period.  This 
new  class  of  peaks  will  not  be  considered  as  true  major 
peaks  but  as  major-minor  peaks.  The  constraints  are  as 
foil  ows : 

a)  The  peak  must  be  a  minor  peak; 

b)  It  must  have  amplitude  at  least  as  great  as  one-fifth 
of  the  maximum  graph  value;  and, 

c)  The  pixel  distance  at  which  this  peak  occurs  must  be 
within  1.5  pixels  of  the  difference  between  2  major 
element  size  peak  distances. 

In  (a)  and  (b) ,  the  peak  amplitude  and  peak-valley 
separations  are  somewhat  constrained.  In  (c) ,  it  is 
required  that  there  be  evidence  that  the  element  spacing 
value  is  indeed  a  candidate  for  the  texture  period. 

In  Appendix  A,  an  effort  is  made  to  derive  some 
indication  of  the  robustness  of  the  peak  classification 
algorithm.  When  working  with  natural  scenes  it  is  likely 


that  the  edge  and  direction  images  will  exhibit  some  form 
of  degradation.  In  low  contrast  images  some  edges  may  be 
missed,  in  noisy  images  edges  may  be  found  where  there  are 
no  cor respond ing  object  edges,  and  at  times  edges  are  out 
of  place  by  a  small  number  of  pixels.  The  effects  of 
these  three  types  of  edge  image  degradations  are  examined. 
It  was  determined  that  for  probability  of  deletion  as 
great  as  .8  the  true  texture  period  can  be  expected  to  be 
classified  as  a  major  peak.  In  all  of  the  test  cases 
considered  for  insertions  and  shifts,  the  peak  occurring 
at  the  true  texture  period  in  the  element  spacing  ERA 
would  have  been  successfully  classified  as  a  major  peak. 
A  more  complete  statement  of  results  can  be  found  in 
Section  A.l  of  the  appendix.  This  study  examined  each 
type  of  degradation  in  isolation.  Future  studies  should 
be  made  to  determine  the  combined  effects  of  these  types 
of  degradations. 


Consider  the  set  of  element  spacing  graphs  in 
Fig.  2-12.  As  would  be  expected  only  one  major  peak  is 
found  in  the  30  degree,  150  degree  and  vertical  scan 
directions  for  both  dark  and  light  objects.  These  are 
found  by  using  the  first  set  of  conditions.  In  all  of  the 
other  graphs  one  would  expect  to  find  that  2  major  peaks 
were  discovered.  This  was  true  of  all  but  the  dark 
element  graph  for  the  60  degree  scan  direction.  The 
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second  peak  is  just  below  the  amplitude  threshold. 
Therefore,  it  is  considered  to  be  only  minor.  After  the 
second  set  of  conditions  were  used,  this  peak  and  one 
other  were  reclassified  as  major-minor  peaks.  The  second 
peak  was  found  at  pixel  distance  2  for  light  objects  in 
the  horizontal  scan  direction.  Neither  of  these  2  peaks 
are  at  the  texture  period  for  their  respective  scan 
directions.  However,  the  peak  which  was  added  for  the  60 
degree  scan  graph  was  clearly  a  strong  multiple  of  the 
actual  texture  period  peak,  the  first  one  in  the  graph. 
Although  the  peak  near  distance  2  of  the  horizontal  light 
element  graph  was  neither  the  texture  period  nor  one  of 
its  multiples,  considering  this  peak  as  a  possible  texture 
period  cause  no  problems.  It  was  weeded  out  in  the  next 
level  of  processing  when  no  strong  evidence  could  be  found 
to  support  it  as  the  horizontal  texture  period. 

In  light  of  the  above  example  it  might  seem  that  peak 
classification  is  unnecessary.  Perhaps  all  element 
spacing  peaks  should  be  considered  in  the  search  for 
texture  period.  This  would  probably  work  very  well  for 
artificially  generated  regular  textures  with  perfect  edge 
and  direction  images.  Every  peak  in  the  element  spacing 
graph  would  either  be  the  period  of  the  texture  or  one  its 
multiples.  Hence,  peak  detection  and  a  simple  repetition 
detection  scheme  would  work  well.  However,  when  working 

91 


c 


with  natural  textures  extra  steps  must  be  taken  to  insure 
that  the  interpretation  being  made  is  backed  up  by  strong 
enough  evidence.  Consider  the  120°  and  150°  ERA  graphs  in 
Fig.  2-19.  These  graphs  were  produced  for  the  grass 
subwindow  of  Fig.  2-18  (a).  Using  the  repetition 
descriptor  algorithm  to  be  defined  in  the  next  section 
without  discriminating  between  the  types  of  peaks 
encountered,  we  would  find  that  grass  is  periodic  in  at 
least  these  2  directions,  a  claim  which  is  not 
observational ly  well-founded.  It  is  therefore  necessary 
to  use  some  form  of  peak  classification  scheme  to  properly 
weight  the  evidence  observed. 

Repetition  Descriptor  Calculation 

In  this  section  the  periodicity  of  the  texture  in  the 
given  scan  directions  is  examined.  At  first,  element 
spacing  information  is  used.  If  a  possible  texture  period 
value  is  found,  then  more  support  is  sought  from  the 
element  size  data.  It  should  be  noted  that  whenever 
multiples  or  peak  repetitions  are  sought  a  margin  of  1.5 
to  the  left  and  right  is  given.  Both  element  spacing  and 
element  size  repetition  descriptors  are  calculated  in  this 
section.  The  process  is  described  below. 

The  first  major  peak  is  considered  as  a  possible 
element  spacing  value.  Checks  are  made  to  determine  if 
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peaks  occur  at  multiples  of  this  value.  Then  the 
appropriate  element  spacing  repetition  descriptor  is 
assigned  as  explained  below. 

The  descriptor  is  assigned  MPK  if  each  multiple  of 
the  value  being  considered  is  the  location  of  a  major 
peak.  At  least  one  repetition  must  be  found. 

The  descriptor  is  assigned  PK  if  at  least  one 
multiple  of  the  value  being  considered  is  the  location  of 
a  major  peak  and  the  rest  are  at  least  minor  peaks,  i.e., 
minor  peaks  or  major-minor  peaks.  Again  at  least  one 
multiple  must  be  found. 

The  descriptor  is  assigned  IC  if  all  of  the 
constraints  of  PK  are  met  except  that  one  of  the  peak 
repetitions  is  missing.  At  least  1  repetition  must  be 
found.  The  descriptor  is  assigned  FAIL  (i.e.,  zero)  if 
none  of  the  above  cases  apply. 

If  the  assigned  descriptor  is  not  FAIL,  all 
repetitions  of  the  peak  under  consideration  are  tagged. 
The  next  untagged  major  peak  is  then  considered.  When  all 
untagged  major  peaks  have  been  considered,  the  untagged 
major-minor  peaks  are  then  examined. 

If  a  non-zero  element  spacing  repetition  descriptor, 
say  Dl,  has  been  assigned  to  any  element  spacing  value,  N, 
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then  attention  is  shifted  to  the  element  size  graph  in 
search  of  further  support.  This  support,  if  it  exists, 
will  be  a  non-zero  element  size  repetition  descriptor. 
The  descriptor  calculation  process  is  described  below. 

Element  spacing  values  are  considered  in  order  of 
decreasing  positive  element  spacing  repetition  descriptor 
value  Dl.  If  peaks  occur  at  locations  M,  M+N,  M+2N,... 
for  element  size  M  and  element  spacing  value  N  then  the 
size  repetition  descriptor,  say  D2,  for  N  is  set  to  MPK, 
PK,  IC ,  or  FAIL  as  explained  below. 

The  descriptor  is  set  to  MPK  if  each  of  M,  M+N, 
M+2N , . . .  is  the  location  of  a  major  peak.  At  least  one 
repetition  must  be  found. 

The  descriptor  is  set  to  PK  if  each  of  M,  M+N, 
M+2N,...  is  the  location  of  a  peak  which  is  at  least 
minor.  Again,  at  least  one  repetition  must  be  found. 

The  descriptor  is  set  to  IC  if  all  the  constraints  of 
PK  are  met  except  that  one  of  the  peak  repetitions  is 
missing.  At  least  1  repetition  must  be  found.  The 
descriptor  is  set  to  FAIL  (i.e.,  zero)  if  none  of  the 
above  cases  apply. 

If  the  descriptor  produced  while  considering  element 
size  M  is  not  equal  to  FAIL  then  repetitions  of  the  peak, 


The  repetition  descriptors  D1  and  D2  associated  with 
element  spacing  values  as  well  as  the  ordered  pairs 
<S1,S2>  associated  with  non-permanently  tagged  element 
size  peaks  are  considered  when  descriptions  are  being 
generated  in  the  interpretation  section  described  below. 


For  the  horizontal  scan  direction  ERAS  of  Fig.  2-12 
the  element  spacing  value  14  has  D1  *  D2  *  MPK  for  both 
dark  and  light  objects.  The  element  size  values  10  (for 
dark  objects)  and  4  (for  light  objects)  have 
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<S 1 , S2>  =  <1 4 , MPK>. 

Description  Generation  Section 

A  simplified  outline  of  this  program  section  is  given 
in  Fig.  2-23.  If  all  of  the  case  statements  and  nested 
conditionals  were  expanded  this  section  would  essentially 
consist  of  a  set  of  production  rules  of  the  form 

IF (CONDI  AND  C0ND2  AND... AND  CONDN) 

THEN  PRINT (Appropr iate  Descriptive  Message). 

A  small  subset  of  these  expanded  production  rules 
follows : 

Rule  1_:  IF(Element  size  graph  closely  approximates  a 
constant) 

THEN  PRINT  ("No  evidence  of  periodicity  or 
predominant  element  size"). 

Rule  2:  IF(Element  spacing  graph  closely  approximates  a 
constant) 

AND  (Element  size  graph  has  constant  profile  0) 
AND  (There  exists  a  major  peak  at  element  size 
value  N  with  amplitude  descriptor  STRONG) 

THEN  PRINT ( "No  evidence  of  periodicity.  STRONG 
evidence  of  element  size  of  N") . 


Rule  3:  IF(Element  size  graph  has  constant  profile  0) 


AND  (Element  size  graph  has  constant  profile  0) 
AND  (There  is  a  major  peak  at  element  spacing 
value,  N,  having  amplitude  descriptor  STRONG) 

AND  (Element  spacing  repetition  descriptor,  Dl, 
for  N  is  MPK) 

AND  (Element  size  repetition  descriptor,  D2,  for 
N  is  MPK) 

THEN  PRINT  ("There  is  VERY  STRONG  evidence  of 
periodicity  with  element  spacing  N") . 

Rule  4:  IF(Element  spacing  graph  has  constant  profile  0) 
AND  (Element  size  graph  has  constant  profile  0) 
AND  (There  is  a  major  peak  at  element  size  value 
N) 

AND  (Amplitude  descriptor  for  N  is  at  least 
STRONG) 

AND  (  N  is  not  permanently  tagged) 

AND  (S2  associated  with  N  is  MPK) 

THEN  PRINT  ("There  is  STRONG  evidence  of  element 
size  N  with  STRONG  support  for  element  spacing 
SI")  . 

For  descriptions  of  Dl,  D2,  SI,  and  S2,  see  the 
repetition  descriptor  calculation  section. 

The  CPU  time  needed  to  compute  the  peak-valley 
signature,  amplitude  and  directionality  descriptors,  and 
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constant  profile  depends  only  on  the  range  of  pixel 
distances  to  be  considered.  However,  the  time  needed  to 
extract  major  peaks,  collect  peak  information,  and 
calculate  repetition  descriptors  depends  on  the  number  and 
types  of  peaks  in  each  graph.  The  time  needed  for 
description  generation  depends  on  the  results  of  the  above 
mentioned  calculations.  The  average  time  recorded  for  31 
pixel  distances  is  .6  seconds.  In  the  following  section 
some  automatically  generated  ERA  descriptions  are  examined 
and  discussed. 

2.4.4  Automatically  Generated  ERA  Descriptions 

Figures  2-24  through  2-36  are  descriptions  of  the 
ERAs  from  Figs.  2-7  through  2-22.  These  descriptions  were 
automatically  generated  by  the  program  described  in 
Section  2.4.3.  They  have  been  shortened  by  excluding  all 
messages  of  the  form:  "No  evidence  of  periodicity  or 
predominant  element  size." 

In  Fig.  2-24,  raffia  is  described  as  a  texture  which 
is  strongly  periodic  in  the  vertical  direction.  Vertical 
element  dimensions  are  given  as  3  for  dark  objects  and  5 
for  light  objects.  There  is  also  strong  evidence  to 
support  the  existence  of  thin  light  elements  found  in  the 
horizontal  scan  direction.  However,  no  period  could  be 
established  for  these. 

V, 


.Pv 
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DARK  OBJECT  DESCRIPTIONS 


In  Figs.  2-25  and  2-26,  there  are  descriptions  for 
the  two  aerial  city  texture  subwindows.  They  both  report 
periodicity,  but  in  different  directions.  The  first  is 
periodic  in  the  horizontal  and  vertical  scan  directions 
while  the  second  reports  periodicity  at  30  degrees.  In 
the  second  description  multiple  element  sizes  are  found  at 
30  degrees  for  both  dark  and  light  objects.  This  is  as 
predicted,  see  Fig.  2-10(b). 

In  Fig.  2-27,  the  herringbone  material  texture  is 
described.  This  texture  is  found  to  be  periodic  in  the 
vertical  and  horizontal  directions.  The  existence  of 
multiple  dark  and  light  element  sizes  in  both  of  these 
scan  directions  is  reported,  see  Fig.  2-11  (b). 

In  Fig.  2-28,  the  floor  grating  image  texture 
description  is  given.  The  30°, 90°,  and  150°  comments  show 
no  evidence  of  periodicity,  while  all  other  directions 
exhibit  strong  evidence  of  a  period  of  approximately  14. 
This  is  due  to  the  fact  that  in  all  other  directions  the 
period  was  too  large  to  show  any  repetitions  of  the 
element  spacing  peak.  Note  that  the  dark  circular  dots 
were  found  as  elements  in  every  scan  direction. 

The  water  and  wood  texture  descriptions  given  in 
Figs.  2-29  and  2-30  are  extremely  similar.  In  both  cases, 
the  texture  are  labelled  as  highly  directional  random 
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Figure  2-28.  Floor  Grating  Texture  Description 
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VERTICAL  SCAN  DIRECTION 


FILENAME  =  WATER 


DARK  OBJECT  DESCRIPTIONS 


patterns  in  the  horizontal  scan  direction  description. 
The  strongest  evidence  reported  is  for  an  element  size  of 
3  in  the  horizontal  scan  direction  for  both  light  and  dark 
objects . 

The  2  straw  images  are  described  in  Figs.  2-31  And 
2-32.  The  major  difference  between  these  2  descriptions 
is  the  fact  that  light  element  sizes  are  reported  in  5  out 
of  S  of  the  scan  direction  descriptions  in  Fig.  2-32.  On 
the  other  hand,  predominant  element  sizes  are  indicated  in 
only  2  scan  directions  in  Fig.  2-31.  Looking  at  the 
corresponding  straw  images  (Fig.  2-13  (a)  bottom  row) 
clarifies  this  question.  The  pieces  of  straw  are  stacked 
in  one  direction  in  the  first  straw  image.  In  the  second 
picture  of  straw,  the  pieces  are  placed  in  a  much  more 
random  fashion.  The  descriptions  are,  therefore,  quite 
reasonable. 

The  remaining  4  texture  images  have  roughly  the  same 
descriptions,  see  Figs.  2-33  through  2-36.  In  all  4 
cases,  there  are  no  texture  periods  given.  No  texture 
period  is  apparent  in  the  corresponding  texture  images. 
The  element  sizes  are  all  roughly  the  same  size  and  they 
occur  in  the  same  directions.  One  noticeable  difference 
in  these  descriptions  is  that  the  grass  image  is  described 
as  a  moderately  directional  random  pattern  while  the 
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FILENAME  =  STRAW 2 


DARK  OBJECT  DESCRIPTIONS 
30  DEGREE  SCAN  DIRECTION 

NO  EVIDENCE  OF  PERIODICITY 

WEAK  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (2.31  ) 

60  DEGREE  SCAN  DIRECTION 

NO  EVIDENCE  OF  PERIODICITY 

STRONG  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (2.31  ) 
LIGHT  OBJECT  DESCRIPTIONS 
HORIZONTAL  SCAN  DIRECTION 

NO  EVIDENCE  OF  PERIODICITY 

STRONG  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (  3.00  ) 

30  DEGREE  SCAN  DIRECTION 

NO  EVIDENCE  OF  PERIODICITY 

MODERATE  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (2.31  ) 
60  DEGREE  SCAN  DIRECTION 

MODERATELY  DIRECTIONAL  RANDOM  PATTERN 
NO  EVIDENCE  OF  PERIODICITY 

STRONG  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (2.31  ) 
VERTICAL  SCAN  DIRECTION 

NO  EVIDENCE  OF  PERIODICITY 

MODERATE  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (  3.00  ) 
120  DEGREE  SCAN  DIRECTION 

NO  EVIDENCE  OF  PERIODICITY 

WEAK  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (2.31  ) 


Figure  2-32.  Straw2  Texture  Description 


FILENAME  =  GRASS 


DARK  OBJECT  DESCRIPTIONS 
HORIZONTAL  SCAN  DIRECTION 

MODERATELY  DIRECTIONAL  RANDOM  PATTERN 
NO  EVIDENCE  OF  PERIODICITY 

MODERATE  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (  3.00  ) 
VERTICAL  SCAN  DIRECTION 

NO  EVIDENCE  OF  PERIODICITY 

WEAK  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (  3.00  ) 
LIGHT  OBJECT  DESCRIPTIONS 
HORIZONTAL  SCAN  DIRECTION 

NO  EVIDENCE  OF  PERIODICITY 

STRONG  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (  3.00  ) 

30  DEGREE  SCAN  DIRECTION 

MO  EVIDENCE  OF  PERIODICITY 

WEAK  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (2.31  ) 
VERTICAL  SCAN  DIRECTION 

NO  EVIDENCE  OF  PERIODICITY 

STRONG  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (3.00  ) 
150  DEGREE  SCAN  DIRECTION 
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WEAK  EVIDENCE  OF  PREDOMINANT  ELEMENT  SIZE  (2.31  ) 


Figure  2-33.  Grass  Texture  Description 
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others  are  not.  It  would  seem  that  this  characterization 
is  quite  reasonable. 

2.5  Summary  and  Conclusions 

In  this  chapter  some  earlier  approaches  to  structural 
texture  feature  description  were  discussed.  The  edge 
repetition  array  method  was  presented  and  examined  in 
detail.  ERA  results  were  displayed  and  discussed  for  a 
set  of  natural  texture  images  containing  both  regular  and 
random  texture  patterns.  A  systematic  process  for 
interpreting  ERA  results  was  presented,  and  both  an 
outline  and  a  detailed  description  of  the  algorithm  were 
given.  Finally,  descriptions  automatically  generated  for 
the  original  set  of  texture  images  were  presented  and 
discussed . 

It  has  been  shown  that  ERAS  can  be  used  to  extract 
textural  information  which  can,  in  turn,  be  automatically 
interpreted  to  produce  structural  texture  descriptions. 
Such  features  as  periodicity,  directionality,  and 
predominant  element  size  of  both  light  and  dark  intensity 
textural  elements  can  be  extracted  from  ERA  results  using 
the  algorithm  described  above.  A  basic  assumption  for  use 
of  this  technique  is  that  textural  elements  are  locally 
bounded  by  pairs  of  oppositely  oriented  parallel  edges. 
If  this  were  not  the  case  then  element  sizes  would  not  be 


found.  An  image  composed  of  perfectly  triangular  objects 
might  produce  a  texture  period  peak  in  some  directions  but 
no  element  size  peaks  for  the  triangular  elements  would  be 
found.  A  similar  problem  occurs  when  a  textural  element 
is  bordered  by  a  lighter  element  on  one  side  and  a  darker 
element  on  the  other.  In  this  case,  the  element  size  for 
the  darkest  element  will  be  found  as  well  as  an  element 
size  for  the  combined  darkest  and  middle  intensity 
element.  However,  the  middle  intensity  element  boundaries 
are  mistakenly  seen  as  period  boundaries.  Extensions  to 
this  basic  method  should  include  mechanisms  to  handle  such 
special  cases. 
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CHAPTER  3 


EXTRACTION  AND  ANALYSIS  OF  TEXTURAL  PRIMITIVES 


3. 1  Introduction 

In  the  previous  chapter  a  technique  for  automatically 
generating  descriptions  of  natural  texture  images  was 
presented.  These  descriptions  provide  regularity 
(periodicity),  direction,  element  size,  and  element 
relative  intensity  information.  The  element  size 


information 

is 

the  dimension  of 

the 

element  in 

the 

particular 

scan 

d  irection 

for  which 

the 

description 

was 

generated . 

Therefore , 

we  have 

only 

one-dimensional 

descriptions  for  2-dimensional  texture  primitives.  In 
this  chapter,  a  technique  for  extracting  and  analyzing 
texture  primitives  is  presented.  The  starting  point  of 
this  process  is  the  texture  description  generated  by  the 
algorithm  discussed  in  Chapter  2.  The  end  product  is  a 
set  of  primitive  masks  (one  for  each  texture  primitive 
type)  and  descriptions  of  each  primitive  type  found.  A 
composite  primitive  image  is  also  produced.  All  of  the 
separate  primitive  masks  are  included  in  this  image. 
However,  the  intensity  of  a  particular  primitive  is  set  to 
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the  average  intensity  found  for  that  primitive.  After  the 
algorithm  is  described  in  detail  the  results  generated  for 
the  texture  images  analyzed  in  Chapter  2  will  be  presented 
and  discussed. 

3.2  Texture  Primitive  Extraction  Methods 

Texture  primitives  have  been  defined  as  line 
segments,  uniform  intensity  regions,  parallelogram  tiles 
or  even  pixels.  The  primitive  extraction  technique  one 
uses  depends  on  the  type  of  primitive  to  be  isolated.  The 
following  is  a  brief  overview  of  some  previously  used 
primitive  extraction  methods. 

Conners  and  Harlow  (12]  define  texture  primitives  to 
be  parallelogram  tiles  which  are  repeated  in  2 
non-coil  inear  directions  to  cover  the  plane.  The  size, 
shape,  and  orientation  of  the  basic  textural  primitive  is 
found  by  examining  the  inertia  measure  values  for  spatial 
gray  level  dependence  matrices  calculated  for  a  number  of 
different  directions.  Valleys  in  these  graphs  correspond 
to  the  texture  period  in  the  given  direction.  By  choosing 
the  2  directions  giving  the  smallest  texture  period  (in 
Euclidean  magnitude)  the  orientation,  size,  and  placement 
rules  of  the  tile  ca.;  be  obtained.  Size,  shape,  and 
orientation  do  not  completely  characterize  the  texture 
tile  or  unit  cell.  A  painting  function  or  tile  pattern 


128 


must  also  be  specified.  One  problem  with  this  method  is 
that  many  different  tiles  or  unit  cells  can  be  found  for  a 
given  texture.  Also,  this  characterization  is  only 
meaningful  when  working  with  periodic  textures. 

The  texture  primitive  representation  used  by  Ehrich 
and  Foith  (17]  does  not  share  this  latter  restriction. 
The  unit  patterns  which  they  extract  are  aggregates  of 
2-dimensional  gray  level  peaks.  The  atomic  regions  are 
light  intensity  areas  bounded  by  darker  intensity  areas  on 
either  side  in  a  given  scan  direction.  More  complex 
regions  are  formed  by  recursive  clustering  using  the 
method  of  relational  *-rees.  One  problem  with  this  method 
is  the  assumption  that  a  textural  element  will  be  centered 
at  an  intensity  peak.  There  may  be  a  problem  if  an 
element  is  relatively  light  in  one  scan  direction  and 
relatively  dark  in  another.  In  this  case  the  element  will 
not  be  at  an  intensity  peak,  it  will  be  at  a  saddle  point. 

In  [18]  Wang,  Velasco,  Wu ,  and  Rosenfeld  describe  and 
contrast  3  different  thresholding  based  primitive 
extraction  methods.  Their  primitives  are  connected 
regions  whose  pixel  intensities  are  above  or  below  a  given 
threshold . 

An  alternative  method  for  characterizing  textural 
elements  is  to  define  them  as  uniform  intensity  regions 
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within  the  texture.  Such  a  character ization  is  used  by 
Tsuj i  and  Tomita  [29],  and  Matsuyama,  Saburi,  and  Nagao 
[20].  Maleson,  Brown,  and  Feldman  [19]  use  a  region 
growing  technique  to  divide  the  image  into  uniform 
intensity  regions  which  are  then  approximated  by  ellipses. 
These  elliptical  texture  primitives  are  then  classified 
with  respect  to  similar  intensity  and  shape 
character i sties. 

The  texture  primitive  characterization  to  be 
presented  here  requires  that  individual  textural  elements 
be  regions  of  like  relative  intensity.  However,  primitive 
intensity  need  not  be  uniform  over  the  entire  primitive 
group.  The  intensity  constraint  used  here  has  to  do  with 
relative  and  not  absolute  intensity.  That  is,  texture 
primitives  of  the  same  type  must  exhibit  the  same  relative 
intensity  with  respect  to  surrounding  regions  in  a 
particular  scan  direction.  Primitive  groups  are  not  only 
able  to  contain  elements  of  various  intensities;  but  they 
are  not  constrained  to  contain  all  uniform  intensity  areas 
exhibiting  the  same  average  gray  level.  This  primitive 
extraction  technique  should  therefore  be  less  sensitive  to 
intensity  differences  due  to  natural  variation  and 
illumination  over  the  texture  images.  Other  constraints, 
e.g.,  element  size  and  orientation  are  also  used.  All  of 
this  information  is  available  in  the  set  of  ERA 
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descriptions  calculated  for  the  texture. 

3.3  The  Primitive  Extraction  Process 
3.3.1  Descriptive  Overview 

The  primitive  extraction  process  to  be  presented  here 
uses  the  one-dimensional  texture  descriptions  generated  by 
the  algorithm  discussed  in  Chapter  2.  For  example,  the 
primitive  search  for  the  raffia  texture  image  of 
Fig.  2-2  (a)  would  begin  with  the  internal  form  of  the 
texture  description  found  in  Fig.  2-24.  One  might  be 
tempted  to  try  to  use  the  individual  components  of  the 
description  to  form  a  two-dimensional  description  of  some 
of  the  textural  elements.  Since  a  light  element  size  of  3 
is  found  in  the  horizontal  scan  direction  and  one  of  5  is 
found  in  the  vertical  scan  direction,  the  existence  of  a 
light  element  with  horizontal  dimension  3  pixels  and 
vertical  dimension  5  pixels  would  be  assumed.  This  is,  in 
fact,  not  the  case.  Fig.  3-1  is  an  abstract 
representation  of  the  basic  raffia  primitive  cluster.  The 
light  element  found  in  the  vertical  scan  direction  is  the 
medium  intensity  block  A  of  Fig.  3-1,  while  the  light 
element  found  by  the  horizontal  scan  is  the  high  intensity 
block  C  in  Fig.  3-1.  If  both  dimensions  of  block  A  were 
found  then  the  vertical  dimension  would  be  characterized 
as  belonging  to  a  light  element  while  the  horizontal 
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dimension  would  be  described  as  belonging  to  a  dark 


element.  Clearly,  a  two  dimensional  description  of  the 


texture  cannot 

be 

constructed 

by 

combining  its 

one- dimensional 

parts. 

However , 

the 

one-d imensional 

descriptions  are  of  use  in  the  primitive  extraction 
process . 


Consider  again  the  texture  description  given  in 
Fig.  2-24.  There  are  3  strong  indications  of  predominant 
element  size.  Light  elements  of  size  5  and  dark  elements 
of  size  3  are  strongly  indicated  in  the  vertical  scan 
direction,  while  light  elements  of  size  3  are  strongly 
indicated  in  the  horizontal  scan  direction.  Each  of  these 
predominant  element  sizes  is  the  location  of  a  strong  peak 
in  an  element  size  edge  repetition  array  for  the  raffia 
image  in  Fig.  2-2  (a) .  Knowing  the  exact  locations,  within 
the  original  texture  image,  where  the  edge  matches 
contributing  to  a  particular  strong  peak  occur  would  be 
useful.  It  would  then  be  possible  to  isolate  those 
regions  being  measured.  The  regions,  or  textural 
elements,  would  then  be  automatical ly  grouped  according  to 
relative  intensity  and  size  with  respect  to  the  given  scan 
direction.  It  is  also  known  that  one  set  of  edges 
bounding  each  element  of  the  group  is  oriented  in  the 
direction  perpendicular  to  the  direction  of  scan.  By 
analyzing  this  set  of  textural  elements,  or  primitives. 
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one  could  determine  the  average  area  and  intensity  of 
members  of  the  group,  and  determine  predominant  element 
placement  c ha r ac te r i s t ics  as  well. 


The  primitive  extraction  process  presented  in  this 
thesis  is  an  implementation  of  the  above  idea.  When 
considering  an  element  size  indicated  for  a  particular 
scan  direction  and  relative  intensity,  the  directional 
scan  is  repeated  for  a  restricted  range  of  distances. 
When  a  match  occurs  the  locations  of  the  matching  edges 
and  their  interior  pixels  are  marked  in  an  intermediate 
mask  image.  After  the  scan  is  completed  the  marked 
primitive  interior  slices  are  expanded  to  their  "natural 
boundaries"  in  a  direction  perpend icular  to  the  original 
scan  direction.  The  criteria  for  expansion  is  based  on 
the  local  pixel  intensities  within  the  original  texture 
image  as  well  as  the  locations  of  edges  within  the 
corresponding  edge  image.  The  final  primitive  mask  image 
contains  the  expanded  primitive  interiors.  The  individual 
elements  are  then  analyzed  to  determine  an  average 
primitive  size  and  intensity  for  the  group  as  well  as 
predominant  inter-element  spacings  for  2  perpendicular 
directions.  If  it  can  be  established  that  the  primitive 
type  under  consideration  forms  part  of  the  background  then 
this  information  is  also  noted.  If  it  is  determined  that 
enough  primitives  of  this  type  have  been  found  to  make 


further  analysis  meaningful  the  primitive  description  is 
catalogued  and  the  primitive  mask  is  saved. 

It  is  conceivable  that  a  particular  primitive  can  be 
detected  in  more  than  one  scan  direction.  When  this 
happens  more  than  one  mask  image  will  refer  to  the  same 
class  of  primitives.  It  is  also  possible  to  find  2 
different  primitive  types  having  the  same  dimension  and 
relative  intensity  in  a  particular  scan  direction.  In 
this  case,  one  primitive  mask  may  contain  primitives  of 
more  than  one  type.  Primitive  masks  are  merged  or 
separated  to  rectify  these  situations. 

At  the  end  of  this  analysis  there  exists  a  set  of 
reduced  primitive  masks  and  their  respective  descriptions. 
In  the  following  section  the  primitive  extraction 
algorithm  will  be  explained  in  detail. 

3.3.2  The  Primitive  Extraction  Algorithm 

The  details  of  the  primitive  extraction  algorithm  are 
given  in  this  section.  This  algorithm  is  composed  of  four 
major  parts  as  follows: 

1)  Ordering  of  the  information  resulting  from  ERA 

interpretation  with  respect  to  relative  significance. 

2)  Creating  the  individual  and  composite  primitive  masks. 
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3)  Analyzing  the  primitive  masks. 


4)  Merging  or  separating  primitive  masks  when  they  are 
sufficiently  overlapped. 

I nformat ion  Ordering 

One  part  of  the  internal  texture  description  which 
was  calculated  but  not  used  before  this  algorithm  is  the 
significance  number.  A  significance  number  is  calculated 
for  each  (intensity,  direction)  pair.  It  is  used  to 
classify  the  12  pairs  of  ERAS  in  terms  of  their  relative 
"importance."  The  importance  of  each  (element  spacing, 
element  size)  ERA  pair  depends  on  the  maximum  peak 
amplitudes  in  each  ERA  and  the  number  of  potential 
matching  edges  found  by  each  directional  scan.  The 
maximum  peak  amplitudes  are  easily  found  and  the  maximum 
normalization  number  for  a  given  ERA  reflects  the  number 
of  properly  oriented  edges  encountered.  The  significance 
number  of  ERA  set,  I,  where  Ie{l , 2, . . . , 12} , is  simply  the 
average  of  the  four  ratios  given  below: 

1)  (Maximum  Amplitude  for  Element  Spacing  ERA 
[I ] ) / (Max imum  Amplitude  Over  All  Element  Spacing  ERAs) 

2)  (Maximum  Amplitude  for  Element  Size  ERA  [I ]) /(Max imum 
Amplitude  Over  All  Element  Size  ERAs) 
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3)  (Maximum  Normalization  Number  for  Element  Spacing  ERA 
(I ] ) /(Max imum  Normalization  Number  Over  All  Element 
Spacing  ERAs) 

(4)  (Maximum  Normalization  Number  for  Element  Size  ERA 
[I ] ) / (Max imum  Normalization  Number  Over  All  Element 
Size  ERAs) . 

The  descriptive  information  calculated  for  each 
(intensity,  direction)  pair  is  considered  for  primitive 
extraction  in  order  of  decreasing  significance  number  of 
the  given  pair.  Both  the  relative  ordering  of  information 
and  the  significance  numbers  themselves  will  be  used  in 
the  classification  scheme  presented  in  Chapter  5. 

Primitive  Mask  C reat ion 

At  this  point  the  (intensity,  direction)  pairs  have 
been  sorted  according  to  significance  number.  The 
descriptive  information  for  each  pair  is  considered  in 
turn.  When  evidence  of  a  predominant  element  size  is 
found  for  a  given  (direction,  intensity)  pair  a  scan 
similar  to  the  original  ERA  scan  is  initiated.  Since  the 
object  of  this  scan  is  to  pinpoint  only  certain  matches  it 
will  differ  from  the  original  ERA  scan  in  the  following 
ways : 
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1)  The  scan  is  over  a  restricted  range  of  distances. 

This  range  will  contain  the  element  size  peak 

distances  with  a  margin  of  one  pixel  in  distance  at 
either  end. 

2)  Only  the  indicated  scan  direction  is  used;  and  matches 
are  sought  only  for  element  sizes  of  the  indicated 
relative  intensity. 

3)  Instead  of  recording  the  edge  matches  in  an  edge 

repetition  array  the  2  matching  edges  are  marked  in  a 
blank  image  in  locations  corresponding  to  their 
locations  in  the  original  edge  image.  The  interior 
points,  points  lying  between  the  2  matching  edges 

along  the  direction  of  scan,  are  also  marked. 

In  Fig.  3-2,  the  marking  scheme  is  illustrated.  Both 
edges  are  marked  by  "E"'s.  The  edge  exhibiting  the  lower 
row  location  (and  if  this  test  fails  the  lower  column 

location)  is  marked  in  a  special  way  as  a  starred  edge. 
The  points  between  the  2  edges  along  the  line  of  scan  are 
marked  as  interior  points,  "I." 

The  image  produced  by  this  process  is  the 

intermediate  primitive  image.  This  image  consists  of 
zeros  except  where  edge  matches  and  their  interior  points 
are  marked  as  in  Fig.  3-2. 


The  next  step  is  the  expansion  of  the  interiors  of 
the  primitive  slices  to  form  a  binary  mask  image  for  the 
primitive  set.  In  the  expansion  process  the  original 
texture  image  and  the  edge  image  are  used. 

The  intermediate  primitive  image  is  searched  until  a 
starred  "E"  is  encountered.  Proceeding  in  the  direction 
of  scan  the  first  interior  point,  "I,"  is  found.  A  "1"  is 
placed  in  the  binary  mask  image  at  this  point  and  the 
texture  image  intensity  of  this  point  is  noted.  This  is 
repeated  until  the  edge  mark  at  the  end  of  the  primitive 
slice  is  encountered.  The  range  of  intensity  values  for 
the  interior  pixels  of  the  primitive  slice  is  now  known. 
At  this  point  the  primitive  interior  can  be  expanded  to 
its  "natural  boundary."  The  natural  boundary  of  a 
primitive  is  made  up  of: 

1)  the  edges  given  in  the  edge  image; 

2)  pixels  whose  intensities  are  outside  of  the  interior 
intensity  range  by  more  than  10  gray  levels;  and, 

3)  the  boundary  of  the  texture  image  itself. 

Expansion  of  the  primitive  slice  proceeds  outward 
from  its  interior  points  in  a  direction  perpendicular  to 
the  direction  of  scan.  Figure  3-3  shows  the  expanded 
primitive  within  the  primitive  mask  image  corresponding  to 
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the  primitive  slice  in  Fig.  3-2.  During  the  primitive 
expansion  process  the  composite  primitive  masks  image  is 
being  updated  to  include  the  current  set  of  textural 
elements.  The  value  of  the  entries  is  the  average 
intensity  of  the  particular  textural  element  being 
expanded . 

After  all  of  the  elements  have  been  expanded  both  the 
primitive  mask  and  the  intermediate  primitive  mask  are 
used  to  estimate  predominant  inter-element  spacing  in  the 
scan  direction  and  in  the  direction  perpendicular  to  the 
direction  of  scan.  Both  of  these  quantities  are  used 
later  in  the  program  when  the  expected  number  of  elements 
of  this  type  is  calculated. 

Primitive  Mask  Analysis 

After  all  the  primitives  of  a  given  type  have  been 
expanded  the  primitive  set  can  be  analyzed.  A  region 
extractor  is  used  so  that  each  individual  texture  element 
can  be  examined  to  determine  its  area  in  pixels  and  its 
average  intensity  value.  At  the  time  of  examination  any 
pixel  sized  holes  found  in  individual  textural  elements 
are  filled  in  the  primitive  mask  and  are  examined  as  part 
of  the  element.  Due  to  the  nature  of  the  primitive 


expansion  technique,  the  primitive  dimension  in  the 
direction  perpendicular  to  the  direction  of  scan  can  be 


approximated  by  dividing  the  primitive  area  by  the  scan 
direction  dimension. 

When  all  of  the  elements  have  been  examined  the 
average  primitive  intensity  value,  size  in  pixels  and 
dimensions  in  2  perpendicular  directions  are  calculated. 
This  information  forms  part  of  the  description  of  the 
primitive  type  defined  by  the  mask  being  analyzed. 

Another  part  of  the  primitive  description  is  the 
background  flag.  The  importance  of  this  flag  is  to 
indicate  that  the  primitive  dimensions  may  not  be 
meaningful.  Also,  when  distances  between  the  centroids  of 
different  textural  elements  are  being  calculated  and 
analyzed  in  the  primitive  placement  rule  detection  section 
(see  Chapter  4)  primitives  of  this  type  are  automatically 
marked  for  exclusion. 

During  primitive  mask  analysis  two  conditions,  either 
of  which  would  indicate  that  the  primitive  under 
consideration  forms  part  of  the  image  background,  are 
watched  for.  The  two  background  tests  are  given  below. 
(Note  that  the  second  dimension  of  a  primitive  is  its 
dimension  oriented  perpend icular  to  the  direction  of 
scan.) 

1)  If  the  ratio  of  (primitive  size  in  pixels)  to  (maximum 


possible  element  dimension  in  the  scan  direction)  for 
any  textural  element  is  greater  than  twice  the  maximum 
possible  dimensional  distance,  given  the  image 
boundaries,  then  this  primitive  forms  part  of  the 
image  background. 

2)  If  at  last  40%  of  the  elements  of  a  given  type  have 
their  second  dimension  greater  than  or  equal  to  95%  of 
the  maximum  possible  dimensional  distance,  given  the 
boundaries  of  the  image,  then  this  primitive  forms 
part  of  the  image  background. 

Consider  the  dark  dot  pattern  in  Fig.  2-8 (a) . 
Obviously,  the  high  intensity  area  surrounding  the  dots 
forms  the  image  background.  Light  intensity  elements  are 
found  in  the  vertical  scan  direction  for  this  texture 
pattern.  They  are  the  spaces  between  vertically 
neighboring  dots.  Expanding  these  primitive  slices  in  the 
horizontal  direction  will  cause  them  to  be  joined  to 
neighboring  strips.  The  resulting  primitive  mask  will 
contain  one  huge  primitive.  Since  the  vertical  element 
dimension  is  obtained  from  the  ERA  description  and  the 
element  area  is  the  actual  pixel  count  the  second 
dimension  calculation  will  produce  an  impossibly  large 
number.  This  condition  would  be  detected  by  the  first 
background  test. 
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The  second  background  test  detects  textural  elements 
which  extend  from  one  end  of  the  image  to  the  other.  An 
example  of  this  type  of  primitive  is  the  vertical  mortar 
strips  in  the  non-shifted  brick  pattern  of  Fig.  1-1). 

The  final  task  to  be  performed  in  this  section  is  to 
determine  if  enough  textural  elements  were  found  to 
consider  this  primitive  type  a  meaningful  component  of  the 
texture . 

If  the  primitive  type  forms  part  of  the  background 
then  a  sample  set  size  of  1  is  sufficient.  In  all  other 
cases  a.  least  10,  and  possibly  more,  textural  elements 
must  be  found.  In  any  case,  30  elements  are  deemed 
sufficient  to  qualify  the  primitive  type.  A  special  floor 
value  is  calculated  to  handle  borderline  cases  with  sample 
set  size  of  less  than  30,  but  at  least  10  elements.  In 
these  cases  at  least  one  third  of  the  expected  number  of 
textural  primitives  must  be  found.  The  expected  number  of 
texture  primitives,  E,  is  calculated  as  follows: 

p  _  __ _ Image  area _ 

*  ~  (DIM1  +  ELSP1)  *  (DIM2  +  ELSP2 )  ' 

where  DIMl  and  DIM2  are  the  primitive  dimensions  in  the 
scan  direction  and  the  direction  perpendicular  to  the 
direction  of  scan,  respectively.  Similarly,  ELSP1  and 
ELSP2  are  the  predominant  inter-element  spacings  in  these 
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2  directions.  The  image  area,  as  well  as  all  of  the 
distances  are  in  pixels. 

As  explained  earlier  background  elements  may  mera,  to 
form  only  one  primitive,  therefore,  a  floor  value  of  1  is 
sufficient  for  them.  Any  primitive  group  with  less  than 
10  elements  is  highly  suspect.  If  the  element  dimensions 
or  spacings  are  so  big  that  not  even  ten  elements  can  fit 
into  the  image,  then  a  window  size  problem  is  indicated. 
If  there  is  no  size  or  spacing  problem  then  the  primitive 
type  is  even  more  questionable  and  should  be  dismissed. 
In  either  case,  the  averages  calculated  for  so  small  a 
sample  set  are  of  little  value.  Between  10  and  30 
primitives  is  still  a  fairly  small  sample  set  size  for 
elements  of  a  texture.  However,  if  this  set  represents  at 
least  one  third  of  the  expected  number  of  primitives  then 
it  is  accepted.  This  range  is  marginal;  but  it  does 
represent  a  substantial  percentage  of  the  expected  number 
of  samples.  30  is  assumed  to  be  a  large  enough  set  size 
so  that  the  average  measurements  calculated  will  be 
meaningful.  These  conclusions  are  based  on  tests  with  a 
wide  range  of  texture  types  for  texture  windows  of 
128  x  128  or  256  x  256  pixels.  If  one  is  dealing  with 
much  larger  texture  windows  then  other  thresholds  might  be 
more  meaningful.  If  the  textures  being  considered  were 
artificially  generated  perfect  patterns  then  the 
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thresholds  discussed  above  would  be  unnecessary.  However, 
when  dealing  with  natural  textures  there  can  be 
considerable  elemental  variation,  hence,  the  need  for  a 
reasonably-sized  set  of  elements. 

If  it  is  determined  that  too  few  samples  have  been 
found  then  the  primitive  expansion  process  is  repeated  for 
the  next  potential  element  size  value  for  the  current 
(intensity,  direction)  pair.  If  the  sample  set  size  is 
acceptable  then  the  ERA  description  for  the  next 
(intensity,  direction)  pair  is  considered.  Only  one 
primitive  type  for  each  (intensity,  direction)  pair  is 
currently  handled.  Extending  the  program  to  handle  more 
than  this  would  be  a  natural  next  step. 

Merging  and  Separation  of  Primitive  Types 

The  final  part  of  the  algorithm  deals  with  merging 
primitive  masks  when  it  is  determined  that  they  are 
defining  the  same  primitive  type;  and  separating  primitive 
masks  when  it  is  recognized  that  more  than  one  primitive 
type  is  being  defined. 

After  a  primitive  mask  is  analyzed  and  accepted  as 
defining  a  bona  fide  primitive  type  it  is  ANDed  with  the 
primitive  masks  of  each  other  exist*. -tg  primitive  type  to 
check  for  overlap.  When  overlap  is  more  than  one  third  of 


146 


the  pixels  covered  by  either  mask  separately,  then  the 
masks  will  be  either  merged  or  separated. 

The  primitive  masks  will  be  separated  only  when  all 
of  the  following  occur: 

1)  Neither  primitive  type  forms  part  of  the  background. 

2)  The  scan  directions  used  to  create  the  2  primitive 
sets  are  perpendicular. 

3)  There  is  a  discrepancy  in  the  average  primitive  areas 
of  the  2  primitive  sets.  (The  average  area  of  the 
smaller  primitive  is  less  than  two  thirds  of  the 
average  area  for  the  larger  one.) 

4)  There  is  a  discrepancy  in  the  average  intensities  of 

the  2  separated  mask  areas.  (The  average  intensity  of 
the  masked  area  defining  the  smaller  primitives 

differs  by  at  least  30  gray  levels  from  the  average 
intensity  of  the  masked  area  defining  the  larger 
primitives  minus  the  area  of  intersection). 

If  all  4  conditions  hold  then  the  mask  containing  the 
smaller  primitives  is  subtracted  from  the  mask  containing 
the  larger  primitives.  Next,  the  average  intensity  and 
element  size  found  for  the  larger  primitive  type  is 
adjusted  using  a  region  extractor  on  the  updated  primitive 


mask.  If  the  4  conditions  do  not  hold  then  the  2  masks 
are  merged  (ORed)  and  their  descriptions  are  linked.  In 
this  case  no  new  primitive  type  is  defined. 

The  requirement  that  the  2  primitive  masks  be  defined 
having  perpendicular  directions  of  scan  before  separation 
is  considered  stems  from  the  particular  type  of  problem 
being  corrected  and  the  nature  of  the  expansion  technique. 
Consider  the  primitives  in  Fig.  3-4 (a) .  Both  type  A  and 
type  B  primitives  have  the  same  relative  intensity  and 
element  size  with  respect  to  the  vertical  scan  direction. 
When  relatively  light  vertical  elements  with  element  size, 
d,  are  extracted  both  element  types  will  be  represented  by 
the  same  primitive  mask.  Clearly,  only  elements  with 
vertical  edges  found  by  a  horizontal  scan  will  be  useful 
in  separating  these  primitives.  The  intensity  requirement 
4)  is  used  to  prevent  texture  primitives  like  the  one  in 
Fig.  3-4  (b)  from  being  subdivided. 

In  the  following  section  primitive  extraction  results 
for  the  texture  samples  analyzed  in  Chapter  2  will  be 
presented  and  discussed.  The  amount  of  CPU  time  needed 
for  the  entire  primitive  extraction  process  depends  on  the 
number  of  potential  primitive  types  and  their  respective 
numbers  of  textural  elements.  Sixteen  128  x  128  pixel 
floor  grating  texture  samples  averaged  approximately  1 


minute  of  CPU  time  each  on  a  DEC  KL-10  processor.  A 
potential  textural  element  was  sought  for  each  intensity, 
direction  pair  for  every  one  of  the  16  subwindows. 


I 


L 


3.4  Primitive  Extraction  Results 

In  this  section  primitive  extraction  results  are 
presented  for  the  texture  samples  discussed  in  Chapter  2. 
Consider  the  raffia  texture  image  in  Fig.  2-2 (a) .  Its 
corresponding  ERA  description  is  given  in  Fig.  2-24. 
There  are  3  strong  indications  of  element  size  in  this 
description.  Dark  elements  of  size  3  are  found  in  the 
vertical  scan  direction.  This  element  type  corresponds  to 
the  B  type  element  of  Fig.  3-1.  Light  elements  of  size  3 
in  the  horizontal  scan  direction,  and  5  in  the  vertical 
scan  direction  are  also  noted.  These  correspond  to 
primitives  of  type  C  and  A  (of  Fig.  3-1),  respectively. 
In  Fig.  3-5,  the  individual  primitive  masks  for  the  3 
raffia  primitives  are  given.  They  correspond  to  the 
primitives  of  type  A,  B,  and  C  (of  Fig.  3-1), 
respectively.  Figure  3-6  (a)  and  (b)  are  the  original 
raffia  texture  image  and  its  composite  primitive  mask. 
The  diagonally  oriented  elements  represent  a  primitive 
type  which  was  rejected  because  too  few  textural  elements 
were  found.  These  elements  were  caused  by  irregularities 
in  the  texture.  (See  Fig.  3-6  (a) ) .  The  composite 
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(a)  Raffia  Texture  Primitive  Type  A 


(b)  Raffia  Texture  Primitive 
Type  B 


(c)  Raffia  Texture  Primitive 
Type  C 


Figure  3-5. 


Raffia  Primitives 
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(a)  Raffia  Texture  Image 


(b)  Composite  Primitive  Mask 
Image  for  (a) 


(c)  Composite  Primitive  Mask 
Image  for  (a)  with 
Rejected  Primitives 
Excluded 


Figure  3-6. 


Raffia  and  Composite  Primitive  Masks 


primitive  mask  would  appear  like  the  image  in  Fig.  3-6  (c) 
if  these  rejected  elements  were  not  included. 

There  is  considerable  intensity  variation  over 
different  primitives  of  the  same  type  in  both  the  texture 
image  and  the  composite  mask  images.  The  intensities  used 
in  these  masks  are  the  average  intensities  for  the 
individual  textural  elements.  In  Fig.  3-7,  the  primitive 
descriptions  for  the  3  raffia  primitive  types  are  given. 
None  of  these  primitives  are  merged  or  separated. 

Only  tabular  primitive  descriptions  and  composite 
primitive  masks  will  be  presented  for  the  rest  of  the 
image  samples.  The  composite  primitive  masks  show  all  of 
the  texture  elements  considered. 

In  Fig.  3-8 (a)  and  ( b) ,  the  periodic  texture  mosaic 
analyzed  in  Chapter  2  is  presented  with  its  corresponding 
set  of  composite  texture  primitive  masks.  The  primitive 
descriptions  for  these  textures  is  given  in  Tables  3-1 
through  3-4. 

The  primitive  descriptions  for  the  first  aerial  city 
texture  image  are  summarized  in  Table  3-1.  Four 
primitives  are  found.  The  fourth  texture  primitive  is  a 
dark  primitive  found  in  the  vertical  scan  direction  with 
dimensions  12  in  the  vertical  direction  and  4  in  the 
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PRIMITIVE  ANALYSIS  FOR  RAFFIA 

RELATIVE  INTENSITY  IS  DARK  DIRECTION  IS  VERTICAL 

PRIMITIVE  NUMBER  IS:  1 

NUMBER  OF  SAMPLES:  104  SIGNIFICANCE  NUMBER:  1.00 

AVERAGE  PRIMITIVE  DIMENSIONS  ARE:  (  2.00  AND  10.71  ) 

AVERAGE  PRIMITIVE  SIZE  IN  PIXELS  IS:  (  20.93  ) 

AVERAGE  PRIMITIVE  INTENSITY  IS:  (  128.37  ) 

PRIMITIVES  REPEAT  AT  ELEMENT  SPACING:  (  8.00  ) 

IN  ABOVE  MENTIONED  DIRECTION 

RELATIVE  INTENSITY  IS  LIGHT  DIRECTION  IS  VERTICAL 

PRIMITIVE  NUMBER  IS:  2 

NUMBER  OF  SAMPLES:  102  SIGNIFICANCE  NUMBER:  .83 

AVERAGE  PRIMITIVE  DIMENSIONS  ARE:  (  4.00  AND  9.49  ) 

AVERAGE  PRIMITIVE  SIZE  IN  PIXELS  IS:  (  37.31  ) 

AVERAGE  PRIMITIVE  INTENSITY  IS:  (  172.48  ) 

PRIMITIVES  REPEAT  AT  ELEMENT  SPACING:  (  8.00  ) 

IN  ABOVE  MENTIONED  DIRECTION 

RELATIVE  INTENSITY  IS  LIGHT  DIRECTION  IS  HORIZONTAL 

PRIMITIVE  NUMBER  IS:  3 

NUMBER  OF  SAMPLES:  68  SIGNIFICANCE  NUMBER:  .48 

AVERAGE  PRIMITIVE  DIMENSIONS  ARE:  (  2.00  AND  7.79  ) 
AVERAGE  PRIMITIVE  SIZE  IN  PIXELS  IS:  (  14.99  ) 
AVERAGE  PRIMITIVE  INTENSITY  IS:  (  190.44  ) 


Figure  3-7.  Primitive  Descriptions  for  Raffia 
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horizontal  direction.  This  primitive  is  first  separated 
from  primitives  of  type  1,  (dark,  horizontal  elements  with 
dimensions  13  and  2)  then  from  the  primitive  of  type  2 
(light,  horizontal  elements  with  dimensions  3  and  27). 
The  primitive  descriptions  for  the  second  aerial  city 
image  are  given  in  Table  3-2.  There  are  two  sets  of  links 
to  finally  produce  3  texture  element  types.  In  Table  3-3 
the  herringbone  material  texture  primitives  are  described. 
There  are  6  separate  primitive  types.  It  seems  that  the 
dark  vertical  and  horizontal  texture  elements  expand  to 
produce  elongated  primitives  oriented  at  45  and  135 
degrees.  These  primitives  correspond  to  the  dark  stripes 
in  the  herringbone  pattern.  The  description  generated  for 
the  grating  primitives  is  quite  lengthy.  (See  Table  3-4). 
Eleven  primitive  masks  and  descriptions  are  generated;  but 
all  of  these  primitives  are  merged  into  2  primitive  types, 
namely,  the  black  dots  and  the  white  background.  New 
primitive  descriptions  are  not  generated  after  merging. 
It  is  enough  to  know  that  all  of  the  merged  texture 
element  descriptions  actually  refer  to  the  same  texture 
primitive  type. 


In  Fig.  3-9  (a)  and  (b)  the  first  non-periodic  texture 


mosa ic 


(from  Chapter  2)  is  presented  with  its 


corresponding  set  of  composite  texture  primitive  masks. 
The  primitive  descriptions  for  these  textures  is  given  in 
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TABLE  3-2.  PRIMITIVE  DESCRIPTIONS  FOR  CITY2 
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Tables  3-5  through  3-8. 

Note  that  the  primitive  descriptions  for  water  and 
wood  given  in  Table  3-5  and  3-6  are  extremely  similar. 
The  one  distinguishing  character istic  seems  to  be  the 
length  of  the  two  types  of  primitives.  The  wood 
primitives  are  considerably  longer  than  the  primitives 
found  for  water.  The  same  comparison  can  be  made  for  the 
original  texture  images  and  for  the  composite  primitive 
masks  as  well. 

In  Tables  3-7  and  3-8,  the  straw  primitive  texture 
descriptions  are  given.  One  obvious  difference  is  that 
the  first  straw  texture  sample  has  primitives  oriented  in 
only  one  direction  while  the  second  straw  sample  has  4 
primitive  types  generated  by  3  different  scan  directions. 
Both  the  texture  and  composite  mask  images  bear  out  the 
directionality  of  the  first  sample  and  the  lack  of 
d irectional i ty  in  the  second. 

In  Fig.  3-10 (a)  and  (b)  the  mosaic  composed  of  the 
most  random  texture  group  to  be  considered  is  presented 
along  with  its  set  of  composite  primitive  masks.  The 
primitive  descriptions  for  these  texture  samples  are  given 
in  Tables  3-9  through  3-12.  The  primitive  descriptions 
for  all  3  texture  types  (grass,  sand,  and  wool)  are  quite 
similar  as  are  the  composite  primitive  masks  in 


Texture  Mosaic  Including  Grass, 
Sand,  Wool  1,  and  Wool  2  Images 


(b)  Composite  Primitive  Images  from  .a) 

e  3-10.  Random  Texture  Mosaic  and  Con 
Primitive  Masks 
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TABLE  3-3.  PRIMITIVE  DESCRIPTIONS  FOR  HERRINGBONE  MATERIAL 
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TABLE  3-9.  PRIMITIVE  DESCRIPTIONS  FOR  GRASS 
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Fig.  3-10 (b).  In  the  following  section  a  comparison  is 
drawn  between  the  primitive  extraction  method  presented  in 
this  thesis  and  a  simple  thresholding  scheme. 

3.5  Thresholding  Versus  ERA  Cues  for 
Primitive  Extraction 

The  texture  primitive  extraction  method  presented  in 
this  thesis  extracts  texture  primitive  sets  according  to 
the  following  criterion.  Elements  belonging  to  the  same 
primitive  class  should  have  similar  relative  intensity 
with  respect  to  surrounding  regions,  similar  size,  similar 
shape  and  similar  orientation.  The  size,  orientation,  and 
relative  intensity  information  used  is  provided  by  the 
descriptors  calculated  in  the  ERA  interpretation  program 
discussed  in  Chapter  2. 

For  artificially  generated  textures  where  all  like 
elements  have  the  same  intensity  a  simple  thresholding 
scheme  would  work  well  for  primitive  extraction.  It  would 
also  be  quite  adequate  for  textures  like  the  floor  grating 
pattern  of  Fig.  2-8  (a)  which  have  high  contrast  and  good 
element  separation.  However,  in  a  low  contrast  texture 
imaqe  like  the  raffia  texture  image  of  Fig.  2-2 (a)  an 
adequate  thresholding  range  might  be  difficult  to  compute. 
In  Fig.  3-11,  a  comparison  can  be  made  between  thresholded 
raffia  image  masks  (a),  (c),  and  (e)  and  the  corresponding 


raffia  primitive  masks  ( b) ,  (d)  ,  and  (f)  created  using 
size,  orientation  and  intensity  cues.  The  thresholding 
ranges  used  for  (a),  (c) ,  and  (e)  were  set  at  the 
approximate  average  primitive  intensity  of  the 
corresponding  primitive  mask  plus  and  minus  10  (gray 
levels).  (All  of  the  texture  images  used  in  this  thesis 
are  8  bit  images.  Therefore,  256  gray  levels  are  possible 
for  each  image.)  The  primitive  regions  in  (b),  (d)  ,  and 
( f )  are  much  more  uniform  than  the  regions  found  in  the 
corresponding  thresholded  image  masks.  Thus,  size,  shape, 
and  placement  analysis  should  produce  better  results  using 
primitive  masks  created  using  this  primitive  extraction 
scheme.  Next,  consider  what  happens  when  the  lower  left 
straw  texture  image  of  Fig.  2-13(a)  is  thresholded  to 
extract  the  light  straw  elements.  Figure  3-12(a)  is  a 
thresholded  version  of  this  image.  It  can  be  seen  that  a 
number  of  the  pieces  of  straw  appear  as  pairs  of  dotted 
lines.  Others  are  blended  with  neighboring  elements.  The 
corresponding  straw  primitive  mask  image  appears  in 
Fig.  3-12(b).  There  is  much  more  uniformity  with  respect 
to  primitive  shape  and  orientation.  Each  element  appears 
to  come  from  a  separate  piece  of  straw  and  not  from  a 
number  of  different  pieces.  Hence,  thresholding  schemes 
may  encounter  difficulties  even  in  high  contrast  images. 
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Figure  3-11.  Raffia  Comparison 


3.6  Summary  and  Conclusions 

In  this  chapter  a  primitive  extraction  algorithm  was 
presented.  The  starting  point  of  the  algorithm  is  the 
internal  form  of  the  set  of  ERA  descriptions  generated  for 
the  given  texture.  Algorithm  details  were  discussed  in 
Section  3.3  and  texture  primitive  masks  and  descriptions 
were  presented  for  a  large  group  of  textures 
(Section  3.4).  In  Section  3.5  the  primitive  extraction 
technique  presented  in  this  thesis  is  compared  to  a  simple 
thresholding  scheme.  Results  seem  to  indicate  that  the 
type  of  information  contained  in  the  ERA  description  is 

valuable  with  respect  to  the  primitive  extraction  process. 

0 
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CHAPTER  4 


AUTOMATIC  DETERMINATION  OF  INTER-PRIMITIVE 
SPATIAL  RELATIONS 


4.1  Introduction 

A  structural  description  for  a  given  texture  should 
consist  of  a  description  of  the  different  types  of 
primitives  comprising  the  texture  and  the  rules  governing 
their  placement.  In  the  previous  chapter  a  scheme  was 
presented  for  extracting  and  analyzing  textural 
primitives.  The  present  chapter  deals  with  the  analysis 
of  their  spatial  relations. 

The  two  types  of  texture  descriptions  discussed  up  to 
this  point  contain  only  partial  primitive  placement 
information.  In  the  ERA  descriptions  only  the  element 
spacing  information  pert?!ns  to  placement.  However,  this 
information  is  only  useful  for  relating  a  primitive  to 
other  primitives  of  the  same  type,  and  in  only  one  scan 
direction.  Similarly,  the  texture  primitive  descriptions 
contain  only  information  relative  to  a  single  primitive 
type.  Consider  the  upper  and  lower  left  brick  subimages 
in  Fig.  4-1  (a) .  It  is  clear  that  the  arrangements  of 


these  2  brick  patterns  are  different.  However,  this 
difference  is  not  captured  in  the  texture  descriptions  of 
Figs.  4-2  and  4-3  or  the  brick  primitive  descriptions  of 


Tables  4-1  and  4-2.  Both  sets  of  bricks  were  found  by 
expanding  elements  discovered  by  the  vertical  scan.  The 
bricks  and  horffzSntal  mortar  strips  separating  them  were 
found  to  be  periodic  in  this  same  direction.  It  is  clear 
that  further  analysis  is  necessary  before  the  spatial 
arrangements  of  these  primitive  sets  can  be  completely 
characterized. 

This  analysis  begins  with  the  set  of  primitive  masks 
created  by  the  primitive  extraction  algorithm  discussed  in 
the  previous  chapter.  These  masks  contain  the  exact 
locations  of  all  the  primitives  of  each  type.  (See  the 
composite  primitive  masks  in  Fig.  4-1 (b) .)  The  method  used 
to  extract  the  primitive  placement  information  from  these 
masks  will  be  discussed  in  detail  later  in  this  chapter. 
In  the  following  section  a  brief  overview  of  other  work 
done  in  this  area  is  discussed. 

4.2  Spatial  Arrangement  Analysis  Methods 

A  number  of  different  methods  for  determining 
inter-primitive  spatial  relationships  will  be  briefly 
discussed  below. 
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DARK  OBJECT  DESCRIPTIONS 


TABLE  4-2.  PRIMITIVE  DESCRIPTIONS  FOR  BRICK3 


Syntactic  approaches  to  texture  analysis  have  been 
explored  by  Carlucci  [13],  Zucker  [16],  Lu  and  Fu  [14], 
[15]  and  others.  Carlucci  [13]  designed  a  graph-like 
language  to  be  used  in  the  generation  of  structural 
descriptions  for  simple  textures.  Zucker  [16]  defines 
textures  in  terms  of  ideal  (perfect)  highly  structured 
patterns  which  undergo  certain  transformations  to  produce 
natural  (observable)  textures.  In  [14]  and  [15]  Lu  and  Fu 
present  a  structural  analyzer  which  uses  tree  grammars  to 
describe  the  spatial  arrangement  of  texture  primitives 
within  square  subwindows  of  a  texture.  The  subwindows  are 
of  a  predetermined  size.  The  texture  elements  are  pixels. 
Like  intensity  pixels  belong  to  the  same  primitive  class. 
The  primitive  types  correspond  to  the  number  of  gray 
levels  found  in  the  image.  On  the  next  level  the  square 
subwindows  must  be  placed  according  to  another  set  of 
syntax  rules.  Recognition  and  synthesis  results  have  been 
shown  for  a  number  of  binary  textures. 

Conners  and  Harlow  [12]  use  the  inertia  measure  of 
the  SGLDM  to  define  the  size,  shape,  and  orientation  of 
the  unit  tiles  comprising  a  texture.  The  SGLDMs  must  be 
calculated  for  angles  between  0°  and  180°  to  determine  the 
direction  in  which  the  smallest  periods  (in  Euclidean 
magnitude)  occur.  The  period  in  a  given  direction 
corresponds  to  the  valleys  of  the  inertia  measure  graph 


for  that  direction.  The  two  smallest  periods  and  their 
corresponding  directions  define  the  underlying  grid 
pattern  of  the  given  texture.  In  order  to  completely 
characterize  the  texture  the  tile  pattern  or  painting 
function  must  then  be  defined. 

In  [21]  Davis  outlines  a  method  for  determining  the 
spatial  arrangement  of  a  texture  pattern  when  the 
underlying  structure  is  a  rectangular  grid.  After  noting 
centroid  locations  for  the  set  of  texture  primitives,  size 
and  direction  histograms  are  calculated.  First,  size 
histograms  are  calculated  for  K  4  nearest  neighbors  to 
determine  the  2  smallest  texture  periods.  These 
correspond  to  the  2  leftmost  peaks  in  the  histogram.  The 
direction  histograms  are  then  calculated  for  the  2  chosen 
distances.  The  final  step  is  to  determine  the  strongest 
pair  of  peaks  (one  from  each  direction  histogram)  such 
that  their  angular  difference  is  approximately  90°. 

Matsuyama,  Saburi,  and  Nagao  [20]  use  "regularity 
vectors"  to  characterize  texture  primitive  arrangement.  A 
regularity  vector  is  similar  to  a  placement  rule.  It 
defines  a  frequently  occurring  relative  vector  between 
elements  in  the  texture.  Along  with  each  regularity 
vector  is  the  set  of  the  individual  element  pairs  related 
by  that  vector.  Initially,  the  vectors  are  defined  only 
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for  like  element  pairs.  Next,  the  regularity  vectors  are 
defined  on  regularity  vector  sets  to  characterize  more 
complex  ar rangements .  Results  were  shown  for  a  number  of 
complex  artificially  generated  textures  and  one 
homogeneous,  regular  natural  texture  sample. 

The  final  spatial  arrangement  analysis  technique  to 
be  discussed  is  the  method  presented  by  Maleson,  Brown, 
and  Feldman  in  [19].  Initially,  regions  of  similar 
intensity  within  the  texture  field  are  approximated  by 
ellipses.  These  elliptical  texture  primitives  are  then 
grouped  into  equal  intensity,  similar  shape  classes. 
Spatial  relations  are  then  defined  amonrj  textural  elements 
within  these  classes.  Primitives  are  considered  to  be 
collinear  if  their  major  axes  line  up  and  parallel  if 
their  minor  axes  are  in  the  same  line.  T-joints  and 
V-joints  are  also  considered  to  be  important  spatial 
relationships . 

In  the  following  section  a  technique  for  extracting 
inter-primitive  placement  rules  is  presented.  In  the 
event  that  the  texture  under  consideration  is  homogeneous 
and  regular  a  minimum  set  of  rules  can  be  extracted  which 
characterize  the  underlying  grid  pattern.  A  comparison 
using  the  original  texture  image  and  a  reconstructed 
texture  image  (using  this  minimum  set  of  rules  along  with 
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a  set  of  average  textural  primitives)  can  be  made. 

The  algorithms  for  minimum  placement  rule  extraction 
and  texture  reconstruction  are  presented  in  section  4.3. 
Texture  reconstructions  and  placement  rule  sets  for  a 
number  of  natural,  regular  textures  are  then  presented  and 
d  iscussed . 

4.3  The  Spatial  Arrangement  Analysis  Method 
4.3.1  Descriptive  Overview 

The  spatial  arrangement  analysis  method  to  be 
presented  here  uses  the  texture  primitive  masks  generated 
by  the  algorithm  discussed  in  Chapter  3.  Clearly,  the 
placement  information  for  all  the  primitives  extracted  is 
preserved  in  their  respective  primitive  mask  images. 
Using  these  primitive  masks  one  can  determine  the  most 
frequently  occurring  spatial  relations  for  all 
combinations  of  non- background  texture  primitive  type 
pairs.  This  set  of  placement  rules  along  with  their 
respective  rates  of  occurrence  within  the  texture  image 
provide  a  reasonable  description  of  the  predominant 
spatial  relations  inherent  in  the  given  texture  pattern. 

If  the  texture  pattern  under  consideration  is 
homogeneous  and  regular  then  this  analysis  can  be  taken  a 
step  further.  A  minimum  set  of  rules  necessary  to 
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characterize  the  texture  pattern  can  be  extracted.  The 
constraints  necessary  for  extracting  this  minimum  set  are 
high  frequency  of  occurrence  and  minimum  Euclidean 
distance.  Without  the  minimum  distance  constraint  the 
underlying  texture  pattern  may  not  be  captured.  Without 
the  high  frequency  constraint  the  rules  selected  may 
represent  an  anomaly  in  the  texture  pattern  rather  than 
the  actual  pattern  being  analyzed. 

The  best  test  of  the  selection  processes  is  to 
compare  the  original  texture  image  with  an  image  created 
using  the  placement  rules  selected  and  a  set  of  average 
texture  primitives.  The  comparison  should  be  made  only 
with  respect  to  the  spatial  relations  of  the  two  images. 
Images  created  using  average  texture  primitives  rarely 
look  exactly  like  the  natural  textures  they  represent. 

In  the  following  subsection  the  details  of  the 
spatial  arrangement  analysis  process  are  given. 


4.3.2  The  Spatial  Arrangement  Analysis  Algorithm 


The  details  of  the  algorithm  used 
underlying  structure  of  a  given  texture 
this  section.  This  algorithm  uses  the 
primitive  masks  created  by  employing  the 
in  Chapter  3.  There  are  3  major  parts  to 


to  analyze  the 
are  presented  in 
set  of  texture 
method  discussed 
this  algorithm: 
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1)  Primitive  Placement  Rule  Generation  -  A  set  of 
predominant  primitive  placement  rules  is  generated  for 
every  possible  pair  of  non- backg round  primitive  types. 

2)  Grid  Relation  Extract  ion  -  If  the  texture  under 
consideration  is  homogeneous  and  regular  then  a 
minimum  set  of  placement  rules,  which  characterizes 
the  texture  pattern,  can  be  extracted. 

3)  Texture  Reconstruct  ion  -  This  algorithm  section  uses 

the  minimum  set  of  placement  rules  produced  in  part  2 
and  a  set  of  average  texture  primitives  in  order  to 
produce  a  reconstructed  texture  image.  The 

reconstructed  texture  image  serves  as  a  means  for 
visually  comparing  the  inferred  set  of  inter-primitive 
spatial  relations  with  the  ones  inherent  in  the 
original  texture  image. 

No  placement  rules  are  generated  for  primitives  which 
form  part  of  the  background.  (See  Chapter  3, 
Section  3.3.2  for  the  details  of  the  background  test.) 
However,  after  the  non-background  primitives  have  been 
placed  in  their  proper  grid  locations  for  the  texture 
reconstruction  the  background  intensity  is  filled  in  if  a 
background  primitive  has  been  indicated  for  the  texture. 
(This  can  also  be  done  before  the  non-background 
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primitives  are  placed.) 


The  thresholds  used  in  the  remainder  of  this  section 
were  chosen  after  numerous  tests  with  many  kinds  of 
texture  samples.  The  need  for  thresholding  is  due  to  the 
fact  that  only  natural  texture  images,  which  are  sometimes 
quite  noisy,  are  used. 

Primitive  Placement  Rule  Generation 

1 )  Primitive  Centroid  Calculation 

Since  the  images  being  analyzed  are  natural 
texture  images  there  can  be  considerable  variation 
among  the  texture  primitives  of  a  given  primitive 
type.  Therefore,  it  is  reasonable  to  define 
placement  rules  in  terms  of  distances  and  angular 
separation  between  the  centroids  of  textural 
elements. 

The  centroid  is  calculated  for  each  individual 
texture  primitive.  The  centroid  locations  are  stored 
in  a  primitive  centroid  list  along  with  the  primitive 
type  of  the  textural  element.  During  the  centroid 
calculation  process  a  new  mask  image  is 
simultaneously  created  for  each  primitive  type.  It 
uses  the  index  into  the  primitive  centroid  list  as 
the  mask  value  for  each  of  its  primitives.  See 
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Fig.  4-4.  This  primitive  index  mask  image,  along 
with  the  primitive  centroid  list  are  all  that  is 
needed  for  the  placement  rule  determination  process. 

Placement  Rule  Determination 

A  set  of  placement  rules  is  generated  for  each 
possible  pair  of  non-background  primitives.  In  order 
to  determine  the  frequency  of  placement  rules 
relating  primitives  of  type  A  to  primitives  of  type  B 
a  12-d irectional  scan  is  initiated  from  each  type  A 
primitive  centroid  location  within  the  type  B 
primitive  index  mask  image.  When  a  primitive  is 
encountered  which  is  different  from  the  primitive  of 
origin,  the  index  value  is  used  to  retrieve  the 
corresponding  centroid  location.  At  this  point  the 
angle  and  distance  between  the  2  primitive  centroids 
are  calculated  and  the  frequency  of  that  particular 
placement  rule  is  increased  by  one.  Placement  rules 
are  noted  to  the  nearest  10  degrees  and  the  nearest  3 
pixels.  Therefore,  when  a  placement  rule  is  observed 
the  frequency  for  the  rule  relating  primitives  of 
type  A  to  primitives  of  type  B  at  the  observed 
distance  divided  by  3  and  the  observed  angle  divided 
by  10  is  increased  by  one. 
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(a)  Primitive  Index  Mask  Image  for  Primitive 
Type  N. 
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(b)  Primitive  Centroid  List  Showing  Entries 
for  (a) . 


Figure  4-4.  Primitive  Indexing  Scheme 
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There  are  2  reasons  for  having  the  placement 
rules  represent  a  range  of  3  pixels  in  distance  and 
10  degrees  in  angular  separation: 

a)  The  first  consideration  has  to  do  with  the  types 

of  images  encountered.  For  noisy  images  like 
raffia,  placement  rules  may  cluster  around  a 
certain  range  of  directions  and  distances  but  a 
significant  amount  of  variation  will  be 

encountered.  Having  larger  angle,  distance  bins 
will  help  to  concentrate  the  results  so  that 
predominant  placement  rules  will  be  more  obvious. 

b)  Another  consideration  is  storage.  The  temporary 

matrix  used  to  store  placement  rule  frequencies 
needs  to  have  N  x  M  x  39  locations,  where  N  is 
the  number  of  primitive  types, 

M  =  Max { (Rows ,  tColumns}/3  +  1,  and  the  angular 
bins  go  from  -19  to  +19  to  provide  the  overlap 
needed  for  further  processing.  When  working  with 
128  x  128  pixel  images  the  requirement  is  1677 
locations  for  every  primitive  type. 

3)  Frequency  Normalization 

The  raw  frequencies  for  the  individual  placement 
rules  will  depend  upon  the  number  of  primitives  of 
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the  type  from  whose  centroids  the  scan  is  being 


initiated,  e.g.,  the  number  of  primitives  of  type  A 
in  the  previous  example.  These  frequencies  also 
depend  upon  the  distance  and  angle  of  the  spatial 
relationship  itself.  Therefore,  the  frequencies  of 
the  individual  placement  rules  will  be  normalized  as 
follows.  Let  F  be  the  number  of  times  the  placement 
rule  relating  primitives  of  type  A  to  primitives  of 
type  B  at  angle  and  distance  D  occurred.  Then 


Normalized (F)  = 


100  x 


,#  Primitives.  (#  Rows-DSin9) (#Cols-DCos9) 
'  of  Type  A  x  (#  Rows  x  #  Cols) 


Normalized  (F)  is  the  percentage  of  time  the  rule 
occurs  given  that  a  primitive  of  type  A  is 
encountered.  Note  that  the  number  of  primitives  of 
type  A  is  multiplied  by  the  ratio  of  the  useful  area 
within  the  image  (given  the  angle  and  distance 
constraint)  over  the  entire  image  area.  In  short,  it 
may  not  be  possible  for  all  of  the  primitives  of  type 
A  to  exhibit  the  particular  inter-pr imitive  spatial 
relationship  under  consideration.  Multiplying  by  the 
above  ratio  should  provide  an  approximate  correction. 


4)  Primitive  Placement  Rule  Selection 


The  final  step  is  the  selection  process.  At 
this  point  a  decision  must  be  made  concerning  the 
relative  importance  of  the  different  placement  rules. 
If  the  image  under  consideration  is  a  perfect 
homogeneous,  regular  texture  image  then  all  of  the 
placement  rules  with  non-zero  frequency  would  have 
percentages  close  to  100%.  Hence,  all  of  them  would 
be  selected  as  members  of  the  final  set.  However, 
when  working  with  natural  texture  images  it  is  not 
unusual  for  a  certain  amount  of  dispersion  to  take 
place.  Under  these  conditions  a  method  for  selecting 
the  most  significant,  or  most  frequently  occurring 
set  of  rules  must  be  developed.  The  selection  method 
used  here  is  outlined  as  follows: 

a)  First,  the  maximum  percentage  of  occurrence,  Ml, 
for  any  placement  rule  for  a  given  primitive 
type,  say  A,  is  noted.  Any  type  A  placement  rule 
with  percentage  at  least  as  great  as  half  of  the 
value  of  Ml,  but  not  less  than  15%  is  kept. 

b)  Any  placement  rule  surviving  the  first  set  of 
thresholds  has  its  percentage  of  occurrence 
replaced  by  the  sum  of  the  nine  matrix  entries. 
These  nine  entries  are  those  whose  degree  and 
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distance  indices  are  different  from  the  original 
entry’s  indices  by  at  most  one.  This  should 
insure  that  any  large  volume  peak  which  is  spread 
out  will  not  be  passed  over  in  the  next 
thresholding  process. 

c)  Next,  the  maximum  percentage  of  occurrence,  M2, 
over  the  new  set  of  entries  is  noted.  Any 
placement  rule  with  a  percentage  at  least  as 
great  as  half  the  value  of  M2,  but  not  less  than 
20%  is  kept  if  it  is  a  local  maximum.  That  is, 
its  value  must  be  at  least  as  great  as  its  eight 
closest  matrix  neighbors. 

d)  If  there  no  remaining  rules  relating  a 

specific  primitive  type  pair  then  the  highest 
placement  rule  relating  these  2  types  is  kept 
even  if  its  percentage  is  less  than  20%. 

The  remaining  placement  rules  belong  to  the 
final  set  of  inter-primitive  spatial  relations  for 
the  given  primitive  type.  These  rules  are  of  the 
form : 

PRIM1  =  A  PRIM  2  =  B  Angle  =  0  DIST  =  D  PER  *  P 

This  relationship  states  that  the  centroid  of  a 
primitive  of  type  A  is  separated  from  the  centroid  of 


a  primitive  of  type  B  by  a  distance  of  D  pixels  at 
angle  9,  P  percent  of  the  time  that  a  primitive  of 
type  A  is  encountered.  This  final  set  of  placement 
rules  is  stored  in  order  of  decreasing  frequency  of 
occurrence . 

Automatic  Grid  Relation  Extraction  for 
Homogeneous  Regular  Textures 

If  we  are  working  with  a  homogeneous  regular  texture, 
i.e.,  a  periodic  texture  in  which  each  primitive  of  a 
given  kind  exhibits  the  same  kind  of  inter-primitive 
spatial  relationships,  then  we  should  be  able  to  extract  a 
minimum,  not  necessarily  unique,  set  of  rules  which 
completely  characterize  the  underlying  texture  pattern. 

In  the  event  that  the  texture  under  consideration  is 
a  striped  pattern  the  program  would  report  that  each 
primitive  type  repeats  in  the  same  direction  and  forms 
part  of  the  background.  Upon  detecting  this  condition  the 
program  would  only  have  to  select  one  placement  rule  for 
each  type  of  primitive.  The  rules  would  relate  each 
primitive  type  to  a  given  primitive  type,  say  A,  in  the 
direction  of  scan  which  is  precisely  the  direction  of 
repetition  for  each  primitive  type  detected. 

The  algorithm  presented  here  was  designed  for 
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homogeneous  regular  textures  in  which  there  is  more  than 
one  direction  of  repetition. 

Suppose  we  are  given  such  a  texture.  If  all  of  the 
primitives  of  one  type,  say  type  A,  are  extracted  a  single 
primitive  "subtexture"  of  the  original  texture  can  be 
formed.  Choosing  the  2  nearest  neighbors  of  a  type  A 
primitive  so  that  the  3  primitives  are  not  collinear,  we 
have  a  configuration  similar  to  the  one  in  Fig.  4-5  (a) . 

This  is  equivalent  to  choosing  2  noncollinear  placement 
rules  relating  primitive  type  A  to  primitive  type  A  which 
minimize  distance.  Since  each  element  of  a  given  type  has 

the  same  environment  the  structure  in  Fig.  4-5  (a)  can  be  * 

propagated  to  form  a  triangular  or  parallelogram  grid 
pattern  as  in  Fig.  4-5 (b) ,  all  of  whose  nodes  correspond 
to  type  A  elements  contained  in  the  subtexture  pattern. 

Suppose  there  exists  a  primitive  of  type  A  in  the 
subtexture  which  is  not  represented  in  the  grid  pattern. 

Placing  this  primitive  in  the  grid,  but  not  in  the 
position  of  a  grid  node  presents  a  problem.  There  is  no 

position  that  this  primitive  can  take  where  it  would  be  at  ■ 

least  as  far  from  every  grid  node  as  the  maximum  distance  ! 

of  the  2  relationships  chosen  above,  and  these  1 

relationships  were  chosen  so  that  inter-primitive  j 

i 

distances  would  be  minimized.  This  is  a  contradiction.  | 

I 

I 
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(a)  2  Non-collinear  Single 
Primitive  Placement 
Rules. 


(b)  Since  each  type  A  primitive 
has  the  same  environment  a 
parallelogram  grid  is 
propagated. 


(c)  Using  an  additional  relationship, 
REL3,  to  add  type  B  primitives 
to  the  grid. 


Figure  4-5.  Texture  Reconstruction  Scheme 


Hence,  there  is  no  primitive  of  type  A  in  the  texture 
which  is  not  included  in  the  grid. 

In  order  to  reconstruct  the  original  texture  pattern 
only  one  placement  rule  need  be  used  to  relate  each 
additional  primitive  type  to  the  grid.  In  each  case  the 
relationship  would  be  between  a  primitive  type  already 
included  in  the  grid  and  a  type  not  yet  included. 

For  example,  suppose  there  exists  a  placement  rule 
relating  type  A  primitives  to  primitives  of  type  B.  This 
rule  could  be  used  to  place  one  type  B  primitive  in  the 
grid  for  each  type  A  primitive  already  included,  see 
Fig.  4-5  (c) .  All  type  B  primitives  placed  in  this  way 
correspond  to  a  type  B  primitives  in  the  original  texture. 
Likewise,  it  can  be  argued  that  there  exists  no  type  B 
primitive  in  the  original  texture  which  is  not  now 
included  in  the  grid.  If  there  were  such  a  primitive  it 
would  not  exhibit  the  inter-primitive  spatial  relationship 
used  to  place  all  of  the  type  B  primitives  already 
included  in  the  grid. 

Hence,  we  need  only  establish  the  underlying  grid 
pattern  using  2  single  primitive  placement  rules,  both 
referring  to  the  same  primitive  type.  Then  only  one 
placement  rule  for  each  additional  non-background 
primitive  type  need  be  used  to  include  the  remaining 
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primitive  types  in  this  grid  structure. 

The  major  steps  in  this  grid  relation  selection 

process  are  as  follows: 

1)  Combine  Redundant  Single  Primitive  Placement  Rules 

Rules  relating  a  primitive  to  itself  at  a 
distance,  D,  and  an  angle, 6  ,  are  combined  with  rules 
relating  this  same  primitive  to  itself  at  distance, 
D,  and  angle  9  +  180  or  6  -  180  . 

2)  Determine  the  Underlying  Grid  Pattern 

From  the  reduced  set  of  single  primitive 

placement  rules  created  in  1  choose  two  rules  to  span 
the  image  space  as  follows: 

a)  Find  the  maximum  percentage  of  occurrence  among 

all  single  primitive  placement  rules.  Any  single 
primitive  rule  with  percentage  at  least  two 
thirds  of  this  value  will  be  considered  a 

candidate  for  the  first  grid  axis. 

b)  Only  those  rules  exhibiting  the  minimum 

inter-primitive  distance  among  the  candidate 
placement  rules  will  be  given  further 


consideration 


c)  The  rule  with  maximum  percentage  of  occurrence 


which  exhibits  the  minimum  inter-primitive 
distance  will  be  chosen  as  the  first  grid  axis. 


d)  The  second  grid  axis  is  chosen  the  same  way  as 
the  first  grid  axis  with  the  exception  that  the 
initial  set  of  candidates  is  somewhat  reduced. 
This  initial  set  of  placement  rules  contains  only 
those  single  primitive  rules  which  pertain  to  the 
same  primitive  used  in  the  first  grid  axis. 
Also,  the  angle  at  which  the  primitives  are 
related  cannot  be  within  10°  of  the  angle  used  in 
the  first  grid  axis  (or  its  opposite). 

Relate  the  Remaining  Primitives  to  the  Grid 


All  non-background  primitives  which  are  not 
specified  in  the  2  grid  axis  placement  rules  must  be 
related  to  the  underlying  grid  pattern.  This  is  done 
as  follows: 

a)  Note  all  placement  rules  which  are  STRONG.  Let 
PI  be  a  placement  rule  relating  a  primitive  of 
type  A  to  a  primitive  of  type  B  at  angle  8  and 
distance  D.  PI  is  STRONG  if  there  exists  a 


placement  rule,  P2,  relating  a  primitive  of  type 
B  to  a  primitive  of  type  A  at  distance  D  and 


angle  8+  180  degrees  or  8-  180  degrees. 


b)  Consider  the  set  of  placement  rules  relating  a 
primitive  included  in  the  grid  structure  to  one 
which  is  not  yet  included.  Note  the  maximum 
percentage  of  occurrence  among  all  of  these 


rules. 

Any 

o  f  the 

rules  whose  percentage 

is 

below 

two 

thirds 

of  the  maximum  value 

is 

dismissed  from  the  group. 

c)  If  any  placement  rules  within  the  new  group  are 
STRONG  then  the  maximum  percentage,  STRONG 
placement  rule  is  chosen  as  the  next  grid 
relation.  If  none  are  STRONG  then  the  only 

constraint  used  will  be  maximum  percentage  of 
occurrence.  The  relationship  chosen  adds  another 
primitive  type  to  the  grid  structure. 

(This  process  is  repeated  until  all  non-background 

primitives  are  included  in  the  grid.) 

Texture  Pattern  Reconstruction 

After  the  grid  relations  have  been  selected  it  would 
be  desirable  to  be  able  to  estimate  how  closely  they 
approximate  the  relations  inherent  in  the  original  texture 
image.  This  can  be  done  by  reconstructing  the  texture 
using  the  selected  grid  relations  and  a  set  of  average 
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texture  primitives.  The  texture  pattern  reconstruction 
process  is  outlined  below. 

1 )  Create  Primitive  Templates 

An  "average"  primitive  mask  is  created  for  each 
non-backg  round  primitive  type  as  follows: 

a)  Using  a  blank  image,  each  primitive  of  a  given 
type  is  superimposed  so  that  their  centers  of 
mass  lie  on  the  same  pixel.  Each  pixel  value 
equals  the  number  of  primitives  covering  that 
pixel . 

b)  All  pixels  with  values  less  than  the  total  number 
of  primitives  of  that  type  multiplied  by  a  user 
specified  threshold  value  are  set  to  zero. 

c)  After  thresholding,  the  reduced  "average" 
primitive  mask  is  stored  and  the  starting  row  and 
column,  the  location  of  the  center  of  mass,  and 
the  maximum  length  and  width  are  recorded. 

2)  Construct  Grid  Skeleton 

Using  the  grid  relation  set,  mark  primitive 
centroid  locations  with  the  primitive  type  number  as 
follows : 
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a)  Starting  at  the  center  of  the  image  the  primitive 
centroid  locations  for  one  axis  are  marked  using 
the  angle  and  distance  information  in  the  first 
grid  axis  placement  rule  as  shown  in  Fig.  4-6 (a) . 

b)  An  axis  of  centroids  is  constructed  through  each 
of  the  first  axis  centroid  locations  using  the 
angle  and  distance  information  in  the  second  grid 
axis  placement  rule.  (See  Fig.  4-6 (b).) 

c)  The  rest  of  the  centroids  are  marked  in  a  similar 
manner.  Using  each  of  the  remaining  placement 
rules  from  the  grid  relation  set,  the  centroid 
locations  of  each  "new"  primitive  type  are 
marked.  That  is,  the  placement  rule  information 
is  used  to  relate  a  primitive  of  one  type  to  a 
primitive  of  another  type  whose  centroids  have 
already  been  marked  in  the  image. 

Fill  in  Primitives  and  Background 

Using  the  average  primitive  templates  created  in 
1  and  the  average  primitive  intensities  calculated 
earlier  ift  the  program  the  texture  pattern  can  be 
reconstructed.  The  reconstructed  texture  image  will 
have  an  "average"  primitive  region  centered  at  each 
of  the  primitive  centroid  markers.  The  intensity  of 
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(a)  First  Grid  Axis  Centroids  for  Placement. 
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(b)  Second  Grid  Axis  Centroids  for  Placement. 

Rule:  PRIM1  =  1  PRIM2  =  1  ANGLE  =  45° 

DISTANCE  =  2/2. 

Figure  4-6.  Grid  Axis  Placement  Scheme 
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this  region  will  be  equal  to  the  average  primitive 
intensity  of  the  respective  primitive  type.  All 
pixels  not  covered  by  a  primitive  will  take  on  the 
background  intensity  if  a  background  has  been 
detected . 

In  the  following  section  the  spatial  arrangement 


analysis 

for  a  number 

of 

periodic  textures 

will 

be 

presented 

and  discussed. 

It 

should  be  noted 

that 

all 

placement  rule  angles  are  assigned  according  to  the  scheme 
shown  in  Fig.  4-7.  Also,  all  single  and  double  starred 
rules  represent  the  first  and  second  grid  axes, 
respectively.  Triple  starred  rules  relate  remaining 
primitives  to  the  underlying  grid  pattern. 

The  amount  of  CPU  time  needed  for  the  spatial 
arrangement  analysis  process  depends  mainly  on  the  number 
of  primitive  types  and  the  number  of  individual  primitives 
per  type.  In  Table  4-3  five  textures  are  compared  with 
respect  to  CPU  time  needed  for  spatial  analysis 
arrangement  processing.  All  samples  are  128  x  128  pixel 
images.  All  processing  was  done  on  a  DEC  KL-10  processor. 

4.4  Spatial  Arrangement  Analysis  Results 

Consider  the  brick  mosaic  of  Fig.  4-1  (a)  .  In  the 
first  brick  subwindow  the  bricks  are  arranged  in  unshifted 
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SPATIAL  ARRANGEMENT  ANALYSIS  TIMING 


rows 


Three  types  of  primitives  are  found  for  this 


texture. 

(See 

Table  4-1.) 

The 

first  primitive  type 

desc  r ibed 

is  the 

horizontal 

mortar  strip. 

The  second 

pr  imitive 

type 

is  the  brick 

pr  imitive. 

The  third 

pr imitive 

type  is 

the  vertical 

mortar  strip 

between  the 

neighboring  brick 

columns. 

The 

vertical 

mortar  strips 

extend  from  one  end  of  the  image  to  the  other.  Hence, 
they  are  labelled  as  background  primitives.  The  placement 
rules  will  then  only  refer  to  the  first  2  primitive  types. 
In  Fig.  4-8  the  placement  rules  for  this  brick  subwindow 
are  given.  The  grid  axes  are  formed  by  the  2  highest 
percentage  single  primitive  placement  rule  sets.  These 
are  the  single  and  double  starred  rules  shown  in  Fig.  4-8. 
(In  some  cases  two  equivalent  rules  were  found  for  each 


g  r  id 

ax  is 

.)  These  2 

sets  of  rules 

refer 

to 

bi 

rick 

pr  im  i 

tives 

.  Another 

rule  is  needed 

to 

related 

the 

hor  i  zontal 

mortar  stri 

p  primitives  to  the 

g  r  id 

of 

br: 

Lck. 

The 

tr  ipl 

e  starred 

rule  is  the  one 

chosen . 

For 

the 

rema  i 

n  ing 

samples  only 

the  underlying  grid  re 

latio 

ns 

and 

1  inks 

will 

be  shown. 

The  grid  relations 

will 

refer 

to 

the 

pi acement 

rules  chosen 

as  grid  axes,  whil 

e  the 

g  r  id 

1; 

inks 

will 

refe 

r  to  the  pi 

acement  rules  selec 

ted  to  rel 

ate 

the 

remaining  primitives  to  the  grid. 


The  upper  right  brick  subimage  in  Fig.  4-1  (a)  is  a 
rotated  version  of  the  upper  left  brick  subimage.  The 
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rotation  is  clockwise  through  90  degrees.  A  comparison 
can  be  made  between  the  texture  descriptions  of  Figs.  4-2 
and  4-9  and  the  primitive  descriptions  of  Tables  4-1  and 
4-4.  The  descriptions  are  nearly  identical  except  for  the 
directional  difference  of  90  degrees.  Likewise,  the 
starred  placement  rules  chosen  for  the  rotated  brick 
(Fig.  4-10)  are  very  similar  to  those  chosen  for  the 
original  (Fig.  4-8)  with  the  exception  of  a  90  degree 
difference  for  each  rule.  The  reconstructions  of  both  of 
these  texture  subwindows  is  given  in  the  reconstruction 
mosaic  of  Fig.  4-11  (b).  (Figure  4-ll(a)  is  the  original 
brick  mosaic.)  Since  a  background  primitive  was  indicated 
for  both  images  their  reconstructed  images  are  completely 
filled  in. 

The  brick  subimage  in  the  lower  left  section  of  the 
brick  texture  mosaic,  Fig.  4-ll(a),  is  a  shifted  brick 
pattern.  Successive  rows  are  shifted  by  half  of  a  brick 
in  distance.  The  texture  descriptions  and  primitive 
descriptions  for  this  image  are  given  in  Fig.  4-3  and 
Table  4-2,  respectively.  The  placement  rules  selected  for 
this  subimage  are  the  starred  rules  of  Fig.  4-12. 

The  highest  percentage  single  primitive  placement 
rule  and  its  counterpart  are  selected  for  the  first  grid 
axis.  However,  the  placement  rule  set  with  the  next 
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(a)  Brick  Mosaic 


(b)  Reconstruction  for  (a' 


Figure  4-11. 
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TABLE  4-4.  PRIMITIVE  DESCRIPTIONS  FOR  BRICK2 


highest  percentage  is  not  chosen.  Instead,  rules  with 
lower  percentage  and  distance  are  selected.  Figure  4-13 
shows  the  underlying  grid  pattern  produced  using  the 
highest  percentage  placement  rules  with  no  attempt  to 
minimize  distance.  This  pattern  looks  similar  to  the 
brick  pattern  shown  in  the  top  row  of  Fig.  4-11  (a).  The 
shifted  rows  are  completely  lost. 

In  the  lower  left  corner  of  Fig.  4-11 (b)  the 
reconstruction  using  the  starred  set  of  placement  rules  is 
shown.  It  appears  that  the  original  subimage  in 
Fig.  4-11  (a)  is  less  slanted  than  the  corresponding 
reconstr uc tion.  This  is  due  to  the  angular  error 
introduced  by  rounding  to  the  nearest  10  degrees.  Instead 
of  -30  degrees,  the  second  grid  axis  rule  should  be 
approximately  -26.6  degrees.  In  an  application,  where 
more  precision  is  necessary,  a  higher  resolution  placement 
rule  search  could  be  implemented.  The  black  gaps  in  the 
reconstruction  exist  because  the  primitive  templates  used 
are  averages  of  the  original  texture  elements.  There  is 
no  guarantee  that  they  will  fit  together  as  precisely  as 
the  originals.  The  lack  of  primitives  around  the  edges, 
and  especially  in  the  lower  right  hand  corner  of  the 
subimage  is  due  to  the  reconstruction  process.  It  if  were 
necessary  to  cover  the  entire  128  x  128  pixel  image  are 
with  the  reconstructed  pattern  this  could  be  done. 
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However,  the  intent  here  is  to  provide  enough  textured 
area  for  a  visual  comparison  of  the  basic  textural 
structure. 

The  last  brick  subimage  in  the  mosaic  is  taken  from  a 
different  brick  wall  image.  It  has  the  same  basic  pattern 
as  the  last  brick  image  analyzed.  The  texture 
descriptions  and  primitive  descriptions  for  this  subimage 
are  given  in  Fig.  4-14  and  Table  4-5,  respectively.  The 
selected  grid  relations  and  grid  links  are  shown  in 
Fig.  4-15.  One  obvious  difference  between  the  2  shifted 
brick  patterns  is  the  difference  in  the  sizes  of  the 
bricks.  There  are  only  10  complete  rows  of  brick  on  the 
right,  as  opposed  to  14  on  the  left.  Each  of  the  rows  on 
the  left  contain  3  nearly  complete  bricks.  On  the  right 
only  5  rows  show  3  complete  bricks.  The  remaining  5  rows 
have  half  bricks  on  either  end.  Since  there  are  fewer 
full  size  primitives  in  the  lower  right  image  the  edge 
effects  will  have  a  much  more  pronounced  effect.  The 
texture  reconstruction  for  this  image  (Fig.  4-11 (b) )  shows 
that  these  effects  produced  enough  confusion  to  inhibit 
the  discovery  of  the  underlying  textural  structure. 

If  the  problem  is,  in  fact,  subwindow  size,  then  a 
larger  window  from  the  same  image  should  be  analyzed 
correctly.  Consider  Fig.  4-16  (a).  The  lower  right 


220 


VERTICAL  SCAN  DIRECTION 
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(b)  Composite  Primitive  Image  (c)  Reconstruction  for  (a) 
for  (a) 


Figure  4-16.  Brick  Texture, Composite  Primitives 
and  Reconstruction 
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TABLE  4-5.  PRIMITIVE  DESCRIPTIONS  FOR  BRICK4 


subwindow  of  Fig.  4-11  (a)  is  the  upper  left  hand  quarter 
of  this  256  x  256  pixel  brick  texture  image.  A  comparison 
can  be  made  of  the  composite  texture  primitive  images  from 
Fig.  4-1 (b)  and  4-16  (b),  and  the  texture  and  primitive 
descriptions  from  Fig.  4-3,  Table  4-2,  Fig.  4-17,  and 
Table  4-6.  Both  texture  descriptions  are  very  similar. 
However,  the  brick  (dark)  element  size  in  the  vertical 
scan  direction  is  larger  by  one  pixel  for  the  256  x  256 
image.  This  is  probably  due  to  a  slight  texture  gradient 
in  the  image.  It  appears  that  the  upper  left  hand  corner 
of  the  wall  was  further  from  the  camera  lens  than  the 
central  or  lower  parts. 

The  2  texture  primitive  descriptions  for  the  brick 
textural  element  show  a  large  discrepancy  (approximately 
80  pixels)  in  average  element  size.  This  is  due  to  the 
fact  that  the  half  bricks  represent  a  much  higher 
percentage  of  the  total  number  of  bricks  in  the  smaller 
sub  image  than  in  the  image  shown  in  Fig.  4-16  (a) .  The  2 
sets  of  grid  relations  and  links  are  shown  in  Figs.  4-15 
and  4-18.  The  correspond ing  texture  reconstruction. 
Figs.  4-11 (b)  and  4-16(c),  are  quite  different.  The 
larger  brick  wall  reconstruction  is  quite  close  to  the 
original  with  respect  to  brick  size  and  relative 
placement.  It  would  seem  that  the  reason  for  the  poor 
reconstruction  in  Fig.  4-11 (b)  (lower  right)  is  the  window 
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TABLE  4-6.  PRIMITIVE  DESCRIPTIONS  FOR  BRICK5 


size  problem 


In  Fig.  4-19,  the  grid  relations  and  grid  links 
extracted  for  the  raffia  image  {Fig.  4-20  (a) )  are  shown. 
The  primitives  referred  to  in  this  list  correspond  to  the 
3  primitive  types  described  in  Fig.  3-11.  The  texture 
reconstruction  is  given  in  Fig.  4-20  (b).  The  unusual 
primitive  shapes  are  a  direct  result  of  the  wide  variation 
in  texture  element  shape  in  the  original  texture  image. 
The  set  of  placement  rules  selected  seem  to  capture  the 
basic  relations  inherent  in  the  abstract  primitive  cluster 
of  Fig.  3-3.  The  elongated  light  primitive  in  the 
reconstruction  seems  to  have  a  varying  spatial 
relationship  with  respect  to  the  broad  light  primitive. 
This  is  due  to  the  fact  that  except  for  certain  directions 
angular  displacements  must  be  approximated  in  a  digital 
image.  The  interactions  of  the  approximations  needed  for 
2  of  the  4  placement  rules  used  help  to  cause  this 
oscillating  effect. 

In  Figs.  4-21  through  4-24  the  sets  of  grid  relations 
and  links  for  the  4  texture  samples  of  Fig.  4-25(a)  are 
given.  The  primitives  refer  to  the  primitives  described 
in  Tables  3-1  through  3-4  except  where  merging  has  taken 
place.  For  example,  the  placement  rules  for  the  floor 
grating  pattern  refer  to  only  1  (non-background)  primitive 
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type.  This  is  because  all  of  the  dark  primitives  were 
recognized  as  belonging  to  the  same  primitive  type  and  so 
were  merged.  Figure  4-25 (c)  is  the  reconstr uction  mosaic 
corresponding  to  the  texture  samples  in  Fig.  4-25(a).  The 
grid  like  appearance  of  the  first  aerial  city  texture 
sample  is  captured  in  its  corresponding  reconstruction. 
The  reconstruction  for  the  second  city  texture  sample 
seems  much  more  abstract.  The  elongated  light  rows  and 
dark  primitives  seem  to  be  well  placed;  but  the 
reconstruction  seems  less  complete  than  for  its  neighbor 
to  the  left.  It  should  be  noted  that  the  reconstruction 
process  has  only  the  extracted  texture  primitive  types  to 
work  with.  These  primitives  are  included  in  the  composite 
primitive  images  of  the  mosaic  in  Fig.  4—2  S  ( b)  .  The 
texture  primitive  composite  mask  image  for  the  rightmost 
aerial  city  image  seems  to  be  less  complete  than  the  one 
on  the  left.  In  this  case,  one  can  only  determine  if  the 
spatial  arrangement  of  the  reconstruction  is  consistent 
with  the  original  texture  image.  It  is  not  meaningful  to 
ask  how  closely  it  duplicates  the  original. 

The  herringbone  material  pattern  is  not  a  homogeneous 
regular  texture  pattern  (as  previously  defined).  It  is, 
instead,  a  texture  made  up  of  alternating  texture  strips. 
All  of  the  texture  primitives  of  a  given  type  do  not  have 
the  same  environment.  Their  environment  depends  on  where 
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they  are  located  in  the  original  texture  image.  The 
minimum  set  of  placement  rules  extracted  do  not  completely 
characterize  the  spatial  arrangement  of  the  primitives 


within 

the  herringbone 

texture 

image. 

Although 

the 

texture 

reconstruction 

might 

look 

similar  to 

the 

herringbone  texture,  it  does  not  capture  the  underlying 
textural  structure  completely. 

The  last  texture  sample  in  this  mosaic  (Fig.  4-25(a)) 
is  the  floor  grating  texture.  Only  2  placement  rules  are 
needed  to  characterize  the  underlying  grid  pattern 
(Fig.  4-24)  since  there  is  only  one  non-background 
primitive  type  after  merging.  The  spatial  arrangement  of 
the  grating  pattern  is  successfully  captured  in  the 
texture  reconstruction.  (See  Fig.  4-25(c).) 

4.5  Summary  and  Conclusions 

In  this  chapter  an  algorithm  for  analyzing  the 
spatial  arrangement  of  the  primitives  comprising  a  given 
texture  was  presented.  The  starting  point  of  the 
algorithm  is  the  set  of  texture  primitive  masks  extracted 
by  using  the  algorithm  discussed  in  Chapter  3.  The 
results  produced  are  a  set  of  primitive  placement  rules,  a 
minimum,  not  necessarily  unique,  set  of  rules  needed  to 
completely  characterize  the  underlying  textural  grid,  and 
a  texture  reconstr uction  image.  The  latter  2,  i.e., 


the 


minimum  set  of  rules  and  the  texture  reconstruction  are 
only  meaningful  when  the  texture  under  consideration  is 
homogeneous  and  regular. 

Results  for  a  number  of  regular  texture  images  were 
presented  and  discussed.  The  textures  samples  analyzed 
include  2  aerial  city  images,  a  raffia  sample,  a 
herringbone  material  subimage,  a  floor  grating  texture 
sample  and  a  number  of  brick  texture  subwindows.  In  most 
cases,  the  underlying  grid  structure  was  captured  by  the 
minimum  set  of  placement  rules  extracted. 
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CHAPTER  5 


STRUCTURAL  TEXTURE  ANALYSIS  APPLICATIONS 

5.1  Introduction 

In  Chapters  2,  3,  and  4,  a  number  of  texture  analysis 
techniques  were  presented.  These  techniques  are  useful 
for  generating  structural  texture  descriptions.  However, 
they  can  also  be  applied  to  other  texture  analysis  tasks. 
Two  such  tasks  are  texture  recognition,  and  surface 
orientation  determination. 

In  Section  5.2  a  texture  classification  scheme  is 
presented.  This  scheme  uses  the  information  contained  in 
the  general  texture  description  produced  by  the  ERA 
interpretation  algorithm.  It  also  uses  the  information 
contained  in  the  individual  primitive  descriptions  created 
by  the  primitive  extraction  algorithm.  These  types  of 
descriptions  were  discussed  in  Chapters  2  and  3, 
respectively.  In  Section  5.2.2,  texture  classification 
results  for  eleven  types  of  textures  are  presented  and 
d iscussed . 

In  Section  5.3  the  problem  of  determining  the 
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orientation  of  textured  surfaces  is  discussed.  The 
discussion  centers  around  the  use  of  the  gradient  of  the 
texture  as  the  primary  surface  orientation  cue.  Work  done 
by  Bajcsy  [5]  and  Stevens  [23],  [30]  in  this  area  is 

briefly  discussed.  An  orientation  analysis  scheme  is 
suggested  which  is  based  on  these  methods  and  inter zo rates 
some  of  the  texture  analysis  techniques  presented  aarlier 
in  this  thesis.  Preliminary  results  for  a  number  of 
texture  images  exhibiting  texture  gradients  are  discussed. 

5.2  Texture  Recognition 

The  texture  recognition  scheme  presented  here  is 
basically  a  decision  tree  classifier.  It  uses  the 
one-dimensional  ERA  i nterpr eta tions  and  the  2-dimensional 
texture  element  descriptions  presented  and  discussed 
earlier.  The  first  describes  a  texture  in  terms  of 
characteristics  which  can  be  detected  from  analyzing  the 
ERA  results  for  a  given  texture  image.  The  second 
describes  classes  of  individual  texture  primitive  types. 
A  texture  recognition  scheme  (for  11  types  of  textures)  is 
discussed  below.  It  uses  the  information  from  both  of 
these  types  of  texture  descriptions. 

5.2.1  Texture  Recognition  Algorithm 

The  structure  of  the  algorithm  used  for  texture 
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classification  is  shown  in  Fig.  5-1.  The  11  texture 
classified  are:  floor  grating  (dark  dot  pattern),  brick 
wall,  aerial  view  of  city,  raffia  (woven  palm), 
herringbone  material,  wood  grain,  aerial  view  of  water, 
straw,  grass,  sand,  and  wool. 

Nost  of  the  textures  used  are  from  the  Brodatz  album 
(22].  The  exceptions  are  the  floor  grating,  brick  wall, 
and  aerial  city  patterns.  Aerial  city  pictures  taken  at 
different  orientations  and  different  scales  were  used. 
(See  the  top  row  of  Fig.  3-8  (a) ) .  Only  11  samples  are 
found.  All  other  texture  groups  consist  of  16  samples 
each.  The  2  basic  brick  wall  patterns  of  Fig.  1-1  are 
used . 


i 

1 

i 

i 
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The  decision  tree  form  was  chosen  for  this  texture 
classi f ication  scheme.  The  structure  of  the  tree  allows 
the  classi f ication  information  to  be  weighted  by  means  of 
relative  ordering.  For  example,  the  test  for  periodicity 
is  encountered  at  an  earlier  stage  than  any  of  the  aspect 
ratio  tests.  Clearly  texture  period  information  is  being 
given  more  significance  than  aspect  ratio  information 
according  to  this  classification  scheme.  An  equivalent 
production  system  or  graph  matching  scheme  could  have  been 
used.  However,  it  was  decided  that  the  tree  structure 
would  be  the  most  efficient  method  to  use  (in  terms  of 
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programming  ease  and  computa tional  complexity)  which  would 
be  adequate  for  classifying  this  set  of  textures. 


The  first  texture  characteristic  considered  is 
periodicity.  If  a  texture  is  non-periodic  it  will  be 
classified  by  the  subtree  to  the  right  of  the  root  node. 
However,  if  the  texture  exhibits  signs  of  periodicity  but 
fails  the  tests  for  each  of  the  5  regular  texture  patterns 
it  is  sent  back  to  the  root  node  re-labeled  as  a 
non-periodic  texture.  Due  to  this  loop  in  the  structure, 
the  classification  algorithm  does  not  have  the  exact  form 
of  a  binary  decision  tree.  However,  for  convenience  tree 
terminology  will  be  used  in  the  discussion.  The  loop  is 
introduced  to  accommodate  textures  which  are  basically 
non-periodic,  but  may  show  evidence  of  periodicity  some  of 
the  time.  The  wood  grain,  water  and  straw  textures 
exhibit  this  charac ter ist ic .  The  opposite  is  not  as 
likely  to  occur,  i.e.,  none  of  the  periodic  textures  are 
ever  mistakenly  sent  down  the  non-periodic  branch. 

No  absolute  texture  element  dimensions  or  intensity 
values  are  used;  and  all  directional  information  is 
relative  as  well.  In  this  way,  the  analysis  should  be 
insensitive  to  scaling,  rotation  and  degree  of  contrast 
within  the  image.  Measures  which  are  used  are  primitive 
eccentricity,  dimension  to  period  ratio,  the  number  and 


significance  of  texture  element  types,  and  relative 

intensities  and  orientations  of  texture  primitive  types. 

The  details  of  each  decision  box  for  the  two  main  decision 

branches  is  given  below: 

Periodic  B ranch 

1)  Dark  Round  Primitives  -  More  than  four  dark 
primitives  are  merged  into  one  primitive  type. 

2)  Multiple  Sized  Primitives  -  Multiple  sized  primitives 
are  found  in  two  pe r pend icul ar  directions.  The 
texture  period  is  equal  to  the  sum  of  the  element 
sizes.  The  sum  of  the  (size/period)  ratios  for  the 
four  most  significant  primitives  is  less  than  .2. 

3)  Most  Significant  Primitives  are  Periodic  in  the  Same 
Direction  -  The  two  most  significant  primitives 
exhibit  an  element  spacinq  value  for  the  same  scan 
d irection. 

4)  Large  (Dark  Element  Size/Light  Element  Size)  Ratio 

The  dark  primitive  is  at  least  twice  as  wide  as  the 
light  primitive  in  the  most  significant  scan 

d irection. 

5)  Light  Elongated  Primitives  -  The  ratio  of  the 

dimension  in  the  most  significant  scan  direction  to 
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the  dimension  in  the  direction  perpendicular  to  the 
direction  of  scan  is  less  than  .4. 

Non-Periodic  B ranch 

1)  Uni-Directional  Texture  -  The  two  most  significant 
primitives  are  found  in  the  same  scan  direction.  The 


significance 

numbers  associated 

wi  th 

these 

two 

primitive 

types  are  greater 

than 

all 

other 

signif icance 

numbers  by  at  least 

.66 

(out 

of  a 

possible  1.0). 

2)  Less  Elongated  Primitives  -  The  sum  of  the  aspect 

ratios  for  the  two  most  significant  primitives  is  at 
least  as  great  as  .175. 

3)  Most  Significant  Primitive  is  Relatively  Dark. 

4)  Elongated  Texture  Primitives  -  The  minimum  aspect 

ratio  of  the  four  most  significant  texture  primitives 
is  no  larger  than  .18. 

5)  Two  Primitives  Types  -  Primitives  are  found  for  the 
two  most  significant  ( intensi ty ,d irection)  pairs. 

6)  Gradual  Loss  of  Significance  -  There  is  no  abrupt 

loss  of  significance  after  the  fourth 

( intensity, direction)  pair.  The  difference  is 
smaller  than  .3. 
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7)  Primitives  Found  in  Two  Perpendicular  Directions. 


8)  Low  Size/Spacing  Ratio  -  There  is  a  low  (element 
size/element  spacing)  ratio  for  relatively  dark 
primitives  in  two  perpend icular  directions.  The  sum 
of  both  ratios  is  less  than  .53. 

9)  Less  Gradual  Loss  of  Significance  -  The  loss  of 
significance  after  the  fourth  ( intensi ty ,d irection) 
pair  is  at  least  .3. 

No  attempt  has  been  made  to  optimize  this  decision 
scheme.  The  classification  results  are  discussed  in  the 
following  section. 

5.2.2  Classification  Results 

Classification  results  are  given  in  the  confusion 
matrix  shown  in  Table  5-1.  The  types  of  samples  to  be 
classified  are  listed  to  the  left  of  the  matrix.  Each  row 
shows  how  a  specific  set  of  samples  was  classified.  For 
example,  15  aerial  water  texture  samples  were  correctly 
classified,  while  one  sample  was  incorrectly  classified  as 
wood  grain.  One  hundred  seventy-one  samples  were 
classified  in  all.  In  most  cases  the  samples  came  from 
512  x  512  pixel  texture  images.  These  were  divided  into 
sixteen  128  x  128  pixel  non-overlapping  texture 
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TABLE  5-1.  CONFUSION  MATRIX 


subwindows.  However,  in  the  case  of  the  aerial  city 
samples  only  If  samples  were  available.  These  were 
cropped  from  two  different  satellite  images  of  the  San 
Francisco  area.  The  16  brick  wall  samples  were  taken  from 
three  separate  brick  wall  images.  One  hundred  fifty-six 
samples  were  correctly  classified  to  give  an  overall 
success  rate  of  91.23%.  It  should  be  noted  that 
additional  contextual  information,  e  ,g . ,  color,  scale,  and 
type  of  scene  would  probably  have  improved  the  results 
obtained.  However,  no  information  of  this  type  was  used 
in  the  classi f ication  scheme  in  order  that  the  strength  of 
the  texture  descriptions  used  could  be  tested  in 
isolation. 

There  are  no  mismatches  for  the  highly  structured, 
regular  texture  group.  However,  there  are  a  number  of 
non-periodic  texture  samples  which  are  classified 
incorrectly.  One  source  of  confusion  is  between  the  water 
and  wood  grain  textures.  Both  are  one-dimensional 
textures  made  up  of  elongated  texture  primitives.  The 
wood  grain  primitives  tend  to  be  more  elongated  than  the 
water  wave  primitives.  There  is  little  else  which  is 
noticeably  different  from  a  structural  point  of  view. 
Hence,  confusion  of  these  two  texture  types  is 
predictable.  Additional  contextual  information,  e.g.,  the 
scale  information  for  both  textures,  would  improve  the 


classification  results. 


Another  area  for  confusion  is  the  set  of  textures  in 
Fig.  3-10  (a).  These  textures  (grass,  sand,  and  wool) 
exhibit  the  least  amount  of  structure  of  the  entire  set. 
The  edge  images  (Fig.  2-18 (b),  ERAS  (Figs.  2-19  to  2-22), 
ERA  descriptions  (Figs.  2-33  to  2-36),  composite  texture 
primitive  masks  (Fig.  3—1 0(b)),  and  texture  primitive 
descriptions  (Tables  3-9  to  3-12)  are  very  similar  for  all 
three  types.  This  is  because  the  program  is  not  designed 
to  measure  the  types  of  features  which  most  readily 
differentiate  these  textures.  In  light  of  these 
description  similarities,  confusion  among  members  of  this 
group  is  to  be  expected.  It  should  be  noted  that  none  of 
these  texture  samples  were  confused  wi  th  textures  from 
outside  of  the  group  and  none  of  the  other  texture  samples 
were  mistakenly  classified  as  one  of  these.  The  same  can 
be  said  of  the  subgroup  formed  by  the  water  and  wood  grain 
textures. 

In  summary  156  samples  out  of  171  were  correctly 
classified  to  give  an  overall  success  rate  of  91.23%  for 
this  classi f ication  scheme.  These  results  are  extremely 
encouraging.  It  would  seem  that  the  information  extracted 
by  the  algorithms  presented  earlier  in  this  thesis 
describe  meaningful  texture  characteristics. 
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5.3  Surface  Orientation  Analysis 

In  Figs.  5-2  and  5-3  two  texture  gradient  images  are 
shown.  Figure  5-2  is  an  image  of  a  brick  wall  and 
Fig.  5-3  is  an  image  of  a  redwood  shake  jof .  In  each 
case  the  textured  surface  is  at  a  non  -  zero  angle  with 
respect  to  the  image  plane.  The  brick  wall  recedes  to  the 
left  of  the  image,  while  the  shake  roof  slants  away  from 
the  viewer  toward  the  top  of  the  image. 

One  cue  which  we  use  to  infer  information  about  the 
orientation  of  textured  surfaces  is  the  texture  gradient, 
i.e.,  the  relative  change  in  size  or  period  of  the 
elements  making  up  the  textured  region  within  the  image. 
Assuming  that  the  textured  surface  being  viewed  is 
homogeneous,  the  element  sizes  of  like  texture  primitives 
should  decrease  with  increased  distance  from  the  viewer. 
The  direction  of  maximum  rate  of  change  of  depth  with 
respect  of  the  observer  should  be  discernible  as  well  as 
the  degree  of  surface  slant  in  this  direction.  A 
convenient  way  to  represent  these  two  surface 
character istics  is  via  the  gradient  space  (as  introduced 
by  Mackworth  [31]). 

Let  z  =  f(x,y)  be  a  function  defining  a  planar 
surface  in  3-Space,  where  the  image  plane  is  parallel  to 
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the  x-y  plane,  (see  Fig.  5-4).  The  surface  normal,  N,  can 


be  defined  as  follows:  N  *  (fx,f^,-l).  Letting  p  *  f^, 
and  q  =  f  ,  we  have  the  gradient  vector,  G  *  (p,q) .  The 
direction  of  G  is  TAN-^ (q/p) ,  while  the  magnitude  of  G  is 
SQRT(p2  +  q2) .  The  direction  of  G  denotes  the  direction 
of  the  greatest  rate  of  change  within  the  image,  while  the 
magnitude  of  G  determines  its  quantity.  Also,  the  tangent 
of  the  angle  that  the  surface  makes  with  the  x-y  (or 
image)  plane  is  equal  to  the  magnitude  of  G.  Therefore, 
the  orientation  of  a  surface  in  3-Space  can  be  represented 
as  a  point  in  the  gradient  space. 

In  [23]  Stevens  suggests  an  alternate  set  of 
coordinates  to  represent  surface  orientation.  He  suggests 
the  pair  (o,x),  where 

o  =  tan-1 ( (p2  +  q2)1/2)  , 

and 

x  =  tan  1(q/p)  , 

Stevens'  slant  and  tilt  angle  terminology  will  be  used. 
The  tilt  angle,  x  ,  is  an  angle  made  with  the  horizontal 
axis  of  the  image  plane.  It  is  the  projection  of  the 
surface  normal  onto  the  image  plane.  Stevens  has  shown 
that  this  direction  coincides  with  the  direction  which 
exhibits  the  greatest  rate  of  change  of  distance  from  the 
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observer  to  the  surface.  This  is  precisely  the  direction 
of  the  texture  gradient.  Therefore,  by  calculating  the 
texture  gradient  one  can  determine  the  tilt  angle.  The 
slant  angle,  o  ,  is  the  angle  which  defines  how  much  the 
image  plane  orientation  differs  from  the  surface  plane 
orientation.  For  a  discussion  of  various  forms  of  surface 
orientation  representation  see  the  works  of  Stevens  [23] 
and  Kender  [32]. 

5.3.1  Methods  for  Determining  Surface  Orientation 

In  [23]  Stevens  proposed  a  method  for  determining  the 
gradient  direction,  or  tilt  angle,  and  the  angle  made  by 
the  surface  normal  and  the  line  of  sight  to  the  surface  at 
that  point.  He  calls  the  latter  the  slant  angle.  They 
are  t  and  a  as  described  above.  (When  the  line  of  sight 
is  the  principle  ray,  then  the  slant  angle  is  equal  to  the 
angle  made  by  the  surface  and  the  image  plane.) 

Calculating  the  tilt  angle  is  fairly  straightforward. 
It  entails  determining  the  direction  of  the  gradient  of 
any  texture  measure  which  is  scaled  (due  to  distance), 
foreshortened  (due  to  surface  orientation),  or  both  scaled 
and  foreshortened. 

Determining  the  slant  angle,  o  ,  is  more  involved. 
One  method  suggested  is  to  determine  which  texture  measure 
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corresponds  to  the  characteristic  dimension.  That  is, 
which  texture  measure  is  scaled  but  not  foreshortened. 
The  normalized  gradient  of  this  dimension,  taken  in  the 
direction  of  the  texture  gradient,  is  equal  to  the  tangent 
of  the  slant  angle. 

IT  ”  tana  '  (5-1) 

where  d  is  the  character istic  dimension.  Characteristic 
dimensions  are  parallel  to  the  image  plane  and  are 
perpendicular  to  the  local  surface  tilt.  Therefore,  after 
the  surface  tilt  has  been  determined  the  orientation  of 
the  cha rac ter istic  dimension  is  known.  This  scheme  for 
calculating  a  cannot  be  used  if  the  image  is  an 
orthographic  projection,  or  if  the  elements  exhibit 
successive  occlusion.  Alternative  schemes  are  explored  in 
[23]  for  handling  these  problems.  Here  it  will  be  assumed 
that  neither  problem  exists. 

In  [5]  Bajcsy  presents  a  method  for  calculating  the 
angle  formed  by  the  image  and  surface  planes.  However, 
the  dimension  used  in  this  case  is  the  dimension  oriented 
in  the  gradient  direction.  Therefore,  it  is  both 
foreshortened  and  scaled. 

Using  the  principles  of  projective  geometry,  the 
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trigonometric  rules  pertaining  to  similar  triangles  and 


some  small  angle  approx imat ions ,  Bajcsy  derives  an 
expression  for  a,  the  angle  formed  by  the  surface  and 
image  planes. 

_ -tana _  _  Fractional  change  in  element  size 
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The  process  has  not  been  fully  defined  or  automated.  An 
outline  of  a  proposed  algorithm  is  discussed  below,  and 
some  preliminary  results  are  presented  in  the  following 
section.  Although  only  part  of  the  program  is 
operational,  see  2  below,  most  of  the  remaining  program 
sections  are  defined  and  seem  feasible  from  a  programming 
point  of  view. 

One  effect  which  must  be  anticipated  by  this 
technique  is  the  possible  changing  orientation  of  the 
texture  primitives.  If  the  texture  gradient  was  strong 
enough  the  height  of  the  brick  primitives  in  Fig.  5-2 
would  be  found  in  the  120  degree  scan  direction  in  the 
lower  left  hand  part  of  the  image  and  in  the  vertical  scan 
direction  in  the  right  and  central  areas.  One  possible 
solution  is  discussed  in  (3)  below. 

A  possible  scenario  for  detecting  surface  orientation 
is  as  follows: 

1.  Divide  the  textured  region  into  locally  uniform 
subwindows,  i.e.,  subwindows  which  exhibit  very 
little,  if  any,  element  size  variation.  This 
algorithm  is  not  yet  defined.  However,  as  a  first 
approx imation  the  image  can  be  divided  into  subwindows 
which  accommodate  the  largest  texture  elements.  Then 
element  sizes  can  be  averaged  over  each  texture 
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subwindow  to  produce  an  element  size  for  that  location 
in  the  image.  This  was  done  manually  for  the  two 
examples  discussed  below.  (In  order  to  automatically 
determine  the  largest  element  size,  one  might 
calculate  modified  ERAS  (see  2  below)  for  a  large 
range  of  distances,  say  up  to  one  third  of  the  largest 
image  dimension,  over  the  entire  texture  image.  Then 
an  ERA  interpretation  routine,  similar  to  the  one 
presented  in  Chapter  2,  can  be  used  to  determine  the 
largest  texture  element  dimensions.) 

2.  Calculate  modified  ERAs  for  each  subwindow  within  the 
region.  Only  the  first  latch  encountered  will  be 
recorded  for  a  particular  directional  scan.  Hence,  no 
element  size  or  spacing  repetitions  should  be  counted. 
It  is  hoped  that  this  will  prevent  repetitions  from 
being  interpreted  as  element  size  variations  within  a 
given  subwindow.  The  modified  ERA  calculation  scheme 
is  operational  and  has  been  used  to  produce  the 
results  shown  in  Fig.  5-5. 

3.  Look  for  a  dimension  exhibiting  strong  results  for 
each  subwindow.  A  dimension,  d,  which  is  locally 
uniform  but  which  reflects  the  gradient  of  the  texture 
is  sought.  It  should  be  verified  that  all  of  the  ERA 
results  chosen  refer  to  the  same  texture  primitive 
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type.  One  way  to  achieve  this  end  is  to  extract 
texture  primitives  for  overlapping  subwindows  and 
verify  that  enough  of  the  primitives  extracted  belong 
to  both  subwindow  neighbors.  In  this  way  textural 
elements  which  shift  orientation  due  to  the  effect  of 
angular  perspective  can  be  found  in  each  subwindow. 

4.  Create  a  matrix,  M,  of  the  centroid  values  of  the 
element  size  peaks  for  d.  Use  a  gradient  operator  on 
M  to  calculate  the  value  of  the  tilt,  or  gradient, 
angle  of  the  surface  within  the  image.  (A  discussion 
of  gradient  operators  can  be  found  in  [33].) 

5.  Knowing  the  tilt  angle  means  that  the  orientation  of 

the  character  istic  dimension  is  also  known.  It  can 
then  be  ascertained  if  either  the  character istic 
dimension  or  the  gradient  dimension  is  already 

available  as  part  of  the  set  of  ERA  results.  If  this 
is  the  case  then  the  rest  of  this  step  can  be  skipped. 
If  this  is  not  true  then  the  character  istic  or 
gradient  dimension  of  some  non-background  texture 
primitive  type  must  be  measured  from  texture  primitive 
masks.  (These  measurements  should  proceed  outward 
from  the  elemental  centers  of  mass.)  Either  of  these 
two  dimensions  can  be  used  for  the  slant  angle 
calculation.  Histograms  of  these  dimensions  should  be 
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kept  so  that  the  final  calculations  can  be  made  using 
the  highest  amplitude/lowest  standard  deviation 
results.  When  measuring  the  characteristic  dimension 
the  original  set  of  subwindows  can  be  used.  However, 
when  calculating  the  gradient  dimension  the  subwindows 
might  have  to  be  recropped  to  provide  the  correct 
center  to  center  angle. 

6.  At  this  point  either  (5-1)  or  (5-2)  can  be  used  to 
calculate  the  surface  slant  angle  to  complete  the 
procedure.  If  the  character istic  dimension  is  known 
then  (5-1)  would  be  used.  If  the  dimension  oriented 
in  the  texture  gradient  direction  is  known  then  (5-2) 
would  be  used.  (See  Section  5.3.1  for  a  brief 

description  of  both  slant  angle  calculation  methods.) 

5.3.3  Texture  Gradient  Examples 

In  this  section  two  texture  gradient  examples  are 
presented  and  discussed.  They  make  use  of  the  general 
method  outlined  in  Section  5.3. 

Example  1 

Consider  the  brick  wall  image  in  Fig.  5-2.  This 
image  is  512  x  512  pixels.  It  was  divided  into  16 
128  x  128  pixel  subimages,  and  ERAs  were  calculated  for 
each  of  these.  The  vertical  element  size  ERAs  for  each 
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subwindow  are  shown  in  Fig.  5-5.  As  expected  the  element 
sizes  decrease  toward  the  left  side  of  the  image. 
Figure  5-6  (a)  shows  the  brick  vertical  element  size 
dimension  matrix.  In  Fig.  5-6 (b) ,  the  results  of  the 
gradient  calculation  are  shown;  and  Fig.  5-6  (c)  shows  the 
results  of  the  tilt  and  slant  angle  calculations.  In  this 
case  the  vertical  brick  dimension  is  the  characteristic 
dimension,  hence,  the  .method  developed  by  Stevens  will  be 
used  to  calculate  the  surface  slant.  The  slant  and  tilt 
calculations  are  carried  out  for  the  4  interior  subwindows 
of  the  image  (Fig.  5-2).  The  tilt  angle  measured  from  the 
image  is  approximately  3°.  The  tilt  angle  results  range 
from  .967°  to  4.52°.  Unfortunately,  the  actual  slant 
angle  is  not  known  for  this  image.  However,  the  results 
found  for  this  angle  seem  to  be  reasonable.  The  angle 
made  by  the  image  and  surface  planes,  or  similarly,  the 
angle  between  the  principle  ray  and  the  surface  normal 
seem  to  be  in  the  neighborhood  of  45°.  In  the  next 
example  the  approximate  angle  formed  by  the  surface  and 
the  image  plane  is  known. 

Example  2 

Consider  the  shako  roof  image  in  Fig.  5-3.  This 
image  is  512  x  512  pixels.  It  was  divided  into  9 


170  x  170  pixel  subimages.  (The  last  two  rows  and  columns 


were  not  used.)  ERAs  were  calculated  for  each  of  the  nine 
subimages.  The  vertical  element  size  ERAS  for  each 
subwindow  are  shown  in  Fig.  5-7.  As  expected  the  element 
sizes  decrease  toward  the  top  of  the  image.  Figure  5-8  (a) 
is  the  matrix  of  the  centers  of  mass  for  the  vertical 
element  size  ERAs  of  Fig.  5-7.  In  Fig.  5-8 (b)  the  results 
of  the  gradient  and  tilt  angle  calculations  are  shown,  and 
Fig.  5-8  (c)  shows  the  results  of  the  slant  angle 
calculation  for  three  image  locations.  The  gradient 
direction  dimension,  i.e.,  the  vertical  dimension,  of  the 
wood  shake  elements  was  already  known  via  ERA  calculation. 
Therefore,  the  the  surface  slant  angle  was  calculated 
using  the  method  developed  by  Bajcsy.  (It  would  be  very 
difficult  to  use  the  characteristic  dimension  in  this 
example  since  the  widths  of  the  wood  shake  are  variable.) 
The  slant  angle  computations  are  carried  out  for  3  sets  of 
data.  The  tilt  angle  is  calculated  once  since  there  are 
only  enough  windows  for  one  gradient  computation.  The 
tilt  angle  for  the  image  (Fig.  5-3)  is  approximately  90°. 
The  tilt  angle  was  computed  to  be  88.49°.  The  slant  angle 
was  calculated  to  be  approximately  71.97°.  (See 
Fig.  5-9.)  The  slant  angle  calculation  results  range  from 
72. 4 05° to  74. 167° 

Determination  of  window  size  is  a  problem  which  must 
be  addressed.  One  possible  solution  is  to  calculate  a  set 


of  ERAs  for  the  entire  image,  initially,  for  a  wide  range 
of  distances.  The  maximum  element  sizes  found  would  then 
dictate  the  appropriate  window  size. 

5.4  Summary  and  Conclusions 

In  this  chapter  the  structural  texture  analysis 
techniques  developed  earlier  were  applied  to  2  texture 
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classification  for  this  group.  In  those  cases  where  there 
was  confusion  additional  contextual  information  would  have 
been  helpful.  For  example,  knowing  scale  information 
would  have  aided  in  distinguishing  wood  grain  from  the 
aerial  water  texture.  As  might  be  expected  the  amount  of 
algorithm  success  varied  directly  with  the  amount  of 
structure  present  in  the  texture.  Since  the  range  of 
textures  handled  by  this  algorithm  is  varied  and  any 
confusion  encountered  is  restricted  to  small  groups,  (2-3) 
of  similar  textures  it  is  fair  to  say  that  the  description 
information  extracted  thus  far  corresponds  to  meaning 
textural  features. 

The  surface  orientation  determination  scheme 
presented  in  Section  5.3  is  in  preliminary  form.  The 
method  described  is  only  partly  automated.  It  uses 
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CHAPTER  6 


r 

? 


SUMMARY  AND  CONCLUSIONS 


In  this  dissertation  a  general  method  for  structural 
texture  analysis  was  presented.  In  chapter  2,  edge 
repetition  arrays  were  defined.  It  was  explained  how  they 
could  be  used  to  extract  certain  structural  information, 
such  as  periodicity,  directionality,  and  predominant 
element  size  from  the  edge  and  direction  images 
corresponding  to  a  natural  texture  image.  An  algorithm 
was  presented  which  automatically  interprets  these  arrays 
and  outputs  a  structural  texture  description.  ERA  results 
were  shown  for  a  number  of  different  natural  texture 
images.  The  texture  descriptions  generated  for  these 
arrays  were  also  presented.  In  most  cases,  the  texture 
periods,  predominant  element  sizes,  primitive 
orientations,  and  relative  elemental  intensities  were 
correctly  reported  in  the  texture  description. 

The  ERAs,  and  hence,  the  texture  descriptions  derived 
from  them  are  produced  by  analyzing  lines  of  pixels  (in  6 
different  directions)  within  the  edge  and  direction 
images.  Therefore,  these  descriptions  are  one-dimensional 
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in  nature.  The  element  sizes  reported  describe  a  single 
dimension  of  the  elements  comprising  the  texture. 
Although  these  texture  descriptions  do  not  provide  enough 
information  in  themselves  to  completely  describe  the 
two-dimensional  texture  elements  they  can  serve  as  the 
starting  point  for  a  two-dimensional  texture  primitive 
search. 

In  Chapter  3,  a  primitive  extraction  algorithm  was 
presented.  It  uses  the  internal  form  of  the 
one-dimensional  texture  description  to  pinpoint  the  actual 
locations  of  the  individual  primitives  within  the  texture 
image.  As  a  result,  individual  and  composite  primitive 
masks  are  produced  along  with  a  set  of  texture  primitive 
descriptions,  one  for  each  texture  primitive  type. 
Texture  primitive  masks  and  descriptions  were  presented 
and  discussed  for  a  number  of  different  types  of  natural 
textures.  The  algorithm  performed  very  well  for  most 
textures  whose  primitives  exhibited  at  least  one  uniform 
d  imension. 

A  structural  texture  description  should  include 
information  pertaining  to  the  arrangement  of  the  elements 
of  which  the  texture  is  composed.  In  Chapter  4,  an 
algorithm  was  presented  which  extracts  the  most  frequently 
occurring  inter-primitive  placement  rules.  In  the  event 
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that  the  natural  texture  being  considered  is  homogeneous 
and  regular,  a  minimum,  not  necessarily  unique,  set  of 
rules  is  extracted  which  completely  describes  the 
underlying  texture  pattern.  As  part  of  the  algorithm,  a 
texture  pattern  is  constructed  using  the  minimum  set  of 
placement  rules  along  with  a  set  of  average  primitive 
templates.  This  texture  reconstruction  can  be  used  to 
visually  determine  how  closely  the  minimum  placement  rule 
set  characterizes  the  underlying  texture  pattern.  Texture 
reconstr uc tions  are  presented  and  discussed  for  all  of  the 
homogeneous,  regular  textures  examined  up  to  this  point. 
In  most  cases,  the  underlying  grid  pattern  was  captured  in 
the  texture  reconstruction.  A  logical  extension  of  this 
algorithm  would  be  to  include  increasingly  complex  texture 
types  in  the  class  of  natural  textures  for  which  a  minimum 
set  of  placement  rules  can  be  extracted. 

In  Chapter  5,  the  texture  analysis  techniques 
presented  in  the  first  4  chapters  are  applied  to  two 
texture  analysis  applications,  namely,  texture  recognition 
and  determination  of  surface  orientation  using  texture 
gradient  cues.  A  texture  classification  scheme  for  the  11 
natural  texture  types  used  in  this  thesis  was  devised.  An 
overall  success  rate  of  91.23%  was  achieved.  Since  the 
range  of  textures  classified  by  this  algorithm  is  varied 
and  any  confusion  which  was  encountered  was  restricted  to 


small  groups  of  similar  textures,  it  would  seem  to 
indicate  that  the  descriptive  information  extracted  by  the 
texture  analysis  scheme  corresponds  to  meaningful  textural 
features. 

A  surface  orientation  detection  scheme  was  outlined 
and  partially  automated.  Preliminary  results  for  two 
images  exhibiting  textured  surfaces  with  non-zero 
gradients  were  presented  and  discussed.  All  known 
quantities  were  calculated  to  within  5°  of  their  actual 
value.  These  results  are  very  promising.  However,  more 
work  needs  to  be  done  in  order  to  fully  automate  the 
surface  orientation  analysis  process. 
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APPENDIX  A 


STATISTICAL  ANALYSIS  OF  ERA  DEGRADATION 

A. 1  Introduction 

In  Chapter  2,  an  ERA  interpretation  algorithm  was 
presented.  The  peak  values  of  every  ERA  are  examined  and 
classified  as  major,  minor,  and  subminor.  The 

classification  decision  depends  on  the  peak  amplitude  and 
the  peak  to  valley  separation.  (See  Chapter  2, 
Section  2.4.3  for  details.)  Given  perfect  edge  and 
direction  images  for  a  periodic  texture  image,  we  would 
expect  to  get  ERA  values  of  100  at  element  size  or  spacing 
distances,  and  zeros  everywhere  else.  However,  the  edge 
and  direction  images  analyzed  are  seldom  free  from  some 
form  of  degradation.  In  low  contrast  images  some  edges 
may  be  missed,  in  noisy  images  edges  may  be  found  where 
there  are  no  corresponding  object  edges,  and  at  times 
edges  are  out  of  place  by  a  small  number  of  pixels.  It  is 
the  object  of  this  study  to  determine  the  robustness  of 
the  ERA  peak  classification  scheme  when  one  of  these  types 
of  degradation  occurs.  In  short,  we  want  to  determine  the 
expected  ERA  values  for  certain  pixel  distances,  assuming 


* 
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a  particular  non-degraded  edge  image  line  and  probability 
of  an  edge  being  deleted,  inserted,  or  shifted. 

Disregarding  the  factor  of  100,  each  ERA  entry  is  the 
number  of  a  particular  type  of  matches  occurring  at  a 
given  pixel  distance  and  for  a  particular  scan  direction 
divided  by  a  similarly  defined  normalization  count.  (See 
Chapter  2  for  details.)  The  number  of  matches  and 
normalization  count  is  calculated  separately  for  each  line 
of  pixels  within  the  image.  Therefore,  this  number  can  be 
thought  of  as  the  ratio  of  two  sums.  For  a  given  line, 
the  number  of  matches  and  normalization  count  are 
correlated.  Only  certain  normalization  counts  can  occur 
given  a  certain  number  of  matches.  However,  the  numbers 
of  matches  (or  normalization  counts)  found  for  separate 
lines  are  independent.  We  want  to  find  the  expected  value 
of  this  ratio. 

The  problem  will  be  modelled  as  the  ratio,  X/Yr  of  2 
correlated  random  variables,  which  are  themselves  sums  of 


independent  random 

variables. 

( 

X  =  Xx  +  X2  +...+  XN  ) 

and  (  Y  =  Y^  +  Y2 

+. . .+  yn  ) , 

where  N  is  the  number  of 

lines  in  the  image. 

and  ,Xi(Yi) 

is 

the  random  variable 

corresponding  to 

the  number 

of 

matches (normal ization 

count)  for  line  i.  The  X.  are  identically  distributed 

l 

because  the  initial  conditions  are  assumed  to  be  the  same 
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for  each  image  line.  This  is  also  true  of  the  Y^. 
Therefore,  we  can  find  the  expected  values  and  (because  of 
independence)  the  variance  for  X(or  Y)  in  terms  of  N  and 
the  expected  value  and  variance  for  X  (or  Y  ).  These 
values  will  be  used  to  calculate  the  expected  value  of  the 
ratio  (X/Y).  See  Section  A. 3. 

The  expected  values,  E  {  X^}  ,  of  the  number  of 
matches  for  a  line  is  calculated  as  follows: 

(For  the  purpose  of  illustration  the  degradation  will 
be  assumed  to  be  edge  deletions.) 

1)  Assume  an  initial  non-degraded  line  configuration, 
that  is,  a  line  with  a  periodic  spacing  of  edges,  and 
a  non-zero  probability,  p,  that  an  edge  is  deleted. 

2)  Given  the  number  of  edges,  E,  and  the  probability  that 
an  edge  is  deleted,  calculate  the  probability  of  zero 
through  E  deletions  occurring  by  using  the  binomial 
density. 

3)  For  each  possible  number  of  deletions  (zero  through  E) 
enumerate  all  of  the  line  configurations  with  that 
number  of  deletions. 


4)  Calculate  P(M|D),  that  is,  calculate  the  probability 
that  M  matches  occur  given  D  edge  deletions.  This  can 


be  calculated  for  D  =  0,1, ...,E  and  M  =  0,1,...,E-1. 

5)  Calculate  P(M),  the  probability  of  M  matches  occurring 
by  forming  the  sum 

P  (M|D=0)P  (D»0)+P  (M  i  D=1 )  P  (D  =  l)  +  .  .  .+P  (M|D=E)P  (D=E)  . 

This  can  be  done  for  M  =  0,1,...,E-1. 

6)  Calculate  the  expected  value,  E  {  }  ,  by  forming  the 

sum  of  the  products  M ( P ( M)  )  for  M  =  0,1,..., E-l. 

The  same  procedure  can  be  followed  to  determine  E  {Y^} 

and  E {  X^}  • 

By  evoking  the  law  of  large  numbers  we  can  use  the 

2 

approximations  for  E{  g(X,Y)  }  and  o  found  in  [34]. 

g(X,Y) 

The  details  of  this  analysis  are  given  in  Section  A. 3. 

The  results  (for  element  spacing  ERAs)  can  be 

summarized  as  follows: 

1)  If  the  only  kind  of  degradation  is  edge  deletions, 
then  the  true  texture  period  can  be  expected  to  be 
classified  as  a  major  peak  for  probability  of  deletion 
as  great  as  .8. 

2)  If  insertions  are  the  only  kind  of  degradation  then 
peak  to  valley  separation  will  decrease  with  increased 
probability  of  an  edge  being  inserted  and  increased 


-T-  r..  -  . 


texture  period.  In  particular,  for  a  texture  period 
of  3,  5,  or  7  pixels  and  probability  of  insertion 
equal  to  .1,  .3,  or  .5  the  true  texture  period  would 
be  expected  to  be  classified  as  a  major  peak. 

3)  If  edge  shifting  is  the  only  source  of  degradation 
then  with  increasing  shift  probabil i ties  the  texture 
period  peak  and  peak  valley  separations  will  decrease. 
For  shift  probabilities  up  to  1/3,  (for  left  or  right 
one  pixel  shifts)  the  texture  period  peak  would  still 
be  correctly  classified  as  major  for  element  spacings 
of  4,  6,  or  3  pixels. 

4)  In  all  cases  the  standard  deviation  was  smaller  (by 
roughly  a  factor  of  ten)  than  the  corresponding 
expected  peak  value. 

In  spite  of  the  fact  that  the  standard  deviations 
were  calculated  along  with  the  expected  peak  values  no 
attempt  was  made  to  calculate  the  precise  probability  of 
correct  peak  classification  in  any  case.  This  is  due  to 
the  fact  that  the  dynamic  thresholds  and  peak  to  valley 
separations  used  in  the  classification  scheme  are  also 
affected  by  the  imposed  degradation.  Thus,  a  more  complex 
study  is  necessary  to  examine  the  combination  of  effects. 
However,  the  results  can  be  used  to  indicate  the  general 
effects  of  such  degradations. 
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A. 2  Assumptions  and  Goals 


This  analysis  is  for  element  spacing  ERAs  calculated 
for  periodic  texture  images.  Hence,  all  matches  are 
assumed  to  be  between  edges  with  exactly  the  same 
orientation.  A  similar  analysis  can  be  carried  out  for 
element  size  ERAs.  The  scan  direction  is  assumed  to  be 
horizontal.  These  results  can  be  applied  to  the  vertical 
scan  direction  by  interchanging  "length  of  line"  with 
"number  of  lines".  However,  analysis  for  other  scan 
directions  would  have  to  take  into  account  the  varying 
line  lengths  for  square  or  rectangular  images.  It  is 
assumed  that  once  a  non-degraded  line  configuration  is 
defined  the  analysis  proceeds  with  no  alterations.  For 
example,  if  the  non-degraded  configuration  of  Fig.  A-l  (a) 
is  assumed,  then  there  can  be  no  insertions  two  pixels  to 
the  left  of  the  first  edge,  even  if  the  rightmost  pixel  is 
deleted  to  keep  the  line  length  the  same.  Although  only 
certain  line  configurations  are  examined  the  method  can  be 
used  to  analyze  arbitrary  non-degraded  line 
configurations. 


Combinations  of  these  effects  are  not  considered. 


In 


Fig.  A-l ,  an  example  of  each  of  these  types  of 
degradations  is  shown.  The  method  used  in  each  case  is 
basically  the  same.  (This  is  the  subject  of  the  following 
section.)  The  primary  reference  is  by  Papoulis  [34]. 

A. 3  General  Scheme 

The  ERA  entry  for  distance,  d,  is  defined  as  follows: 

Number  of  matches  at  distance  d 
x  Normalization  count  for  d  '  (A-l) 


Since  each  line  is  considered  independently, 
rewrite  (A-l)  for  distance  d  as  follows: 

N 


100  x 


£ 

i=l 

N 

£ 


Number  of  matches  for  line  i 


Normalization  count  for  line  i 


i=l 


we  can 


(A-2) 


where  N  is  the  number  of  lines  in  the  image.  (The  factor 
of  100  will  be  ignored  for  the  rest  of  this  analysis.)  The 
aim  here  is  to  derive  an  expression  for  the  expected  value 
of  second  factor  in  (A-2). 


Let  S  be  the  space  of  all  possible  degraded  line 
configurations  for  a  given  initial  non-degraded  line  and 
non-zero  probability  of  one  kind  of  degradation.  For 
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example,  if  the  degradation  is  deletions,  then  S  is  the 
set  containing  the  initial  line  configuration  with  all 
possible  numbers  of  deletions,  from  no  deleted  edges  to 
all  deleted  edges.  In  this  case  S  =  {11,12,...,1  £}  , 
where  E  is  the  number  of  equally  spaced  edges  in  the 
non-degraded  line.  Let  X^  be  the  random  variable  (r.v.) 
associated  with  line  i  in  the  image  which  assigns  each 
line  configuration,  1 j ,  its  number  of  matches  for  some 
predetermined  element  spacing  distance.  For  example,  if 
lj  is  the  degraded  configuration  of  Fig.  A-l  (b)  then 
X^(lj)  =  4  for  element  spacing  3  for  all  i. 

Let  P  (X^  =  k)  be  the  probability  that  a  particular 
line  configuration  with  non-zero  probability  of 
degradation  will  have  k  matches  at  a  given  element 
spacing.  Then 


E(xi)  =  n 

x,  -  Ekjp,xi  - 
j 

V 

(A— 3 ) 

is  the  mean  or  expected  value  of 

X 

and 

(k.-nY  )2P(X.  = 

3  xi  1 

V 

(A-4) 

is  the  variance 

or  dispersion 

o  f 

xi* 

The 

number  of 

matches  for 

each  line  in 

the 

image 

is 

calculated 

independently. 

Therefore, 

the 

random 

var iables 
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are  independent. 


Since  all  of  the  initial 


be  a  function  of  the  random  variables  X  and  Y 


The 


statistics  of  g  are  the  object  of  this  analysis.  That  is, 
we  are  interested  in  the  expected  value  and  standard 
deviation  of  g(X,Y).  (The  standard  deviation  is  simply 
the  square  root  of  the  variance) . 


Since  we  are  working  with  large  sums  of  identically 
distributed  random  variables,  it  is  reasonable  to  assume 
by  the  law  of  large  numbers  that  the  joint  density  is 
concentrated  near  its  mean,  ^n^riy)*  Then  since  (X/Y)  is 
smooth  in  the  regions  of  interest,  the  following 
approximation  from  [34, p. 212]  can  be  used: 


/  3 ^  3^  3^  \ 

e{9<x,m>»9<nx,ny>+  +  '  <A-l0> 


3X 


3X3Y 


3Y 


and 


°g(X,Y)'“(3X)  U20 


+  <^)2 


3Y  02 


3  _  3 

+  2  — 2  — 2  u 
*  3X  3Y  M11 


(A-ll) 


where 


*11 


=  E { XY }  -  E{X}E{Y} 


(A-12) 
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U20  °X  XA  ' 


(A-l 3 ) 


w02  “  °Y  =  N%  * 


(A-l 4 ) 


We  would  like  a  simpler  expression  for  the  covariance 

of  X  and  Y.  By  substituting  in  (A-12)  we  get 


N  N 


\ 

2XZ>, 

1=1  j=l  > 


-NnY  Nnv 
xo  *0 


(A-l 5 ) 


( A— 1 5 )  can  be  rewritten  as  follows 


N  N 


( A-16 ) 


The  double  sum  can  be  broken  into  2  sums  to  give 


“a  =  E 


2XyiH  Z  xiYj}  -n 

i=l  ’  li,jeIN  * 


-  n  n  n  / 


xo  Yo 


(A-l 7 ) 


where  IN  is  the  set  (1,2, The  sum  on  the  right  is 
the  sum  of  products  of  independent  random  variables  since 
the  number  of  matches  and  normalization  numbers  are 
constrained  to  be  from  2  different  lines  in  the  image. 
This  means  that  (A-17)  can  be  rewritten  as  follows 


-  Vtlxx)  +  <N2-mn„  n„  -nV  nY  .  (A-ie) 


Simplifying,  we  obtain 


y  =  N (£{X .  Y.  }-  n  nY  ) 

11  1  x  Ao  0 


(A-19) 


Now  (A-10)  can  be  rewritten  as  follows 


2  2 


E(g(X,Y)  hr - -  - 

Nn 


NnXrt  N(E{xiYi}nx  nY  )  n  n 


0  0 


0  *0 


(Nnv  ) 


(Nnv  ) 
*0 


( A-20 ) 


and  further  simplified  as  follows 


nv  (e{x.y. }-nv  nv  )  oZ  nv 
xo  1  1  xo  Yo  Yo  Ao 

E{g  (X,  Y)  }  rs  - 5 - 0__0_  +  o__o_  #  (A_21) 


Nn 


Nn; 


Finally,  we  have 


_^0  _  /E{ x  Y  _  W\  1 

E(y>-  n„  V{ii>  xo %  "v„  /  N„a  ' 


(A-22) 


where  N  is  the  number  of  lines  in  the  image,  n  is  the 

xo 

expected  value  of  the  number  of  matches  on  a  line,  nv  is 

0 

the  expected  value  for  the  normalization  count  on  a  line. 


and 


is  the  variance  for  the  normalization  count. 


YS  t*ie  expected  value  of  the  product  of  the 
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§r.  twu***- 


All  of 


matching  and  normalization  numbers  for  a  line, 
these  will  be  calculated  by  enumeration  for  a  given 
non-degraded  line  configuration,  element  spacing  distance 
and  probability  of  degradation.  A  similar  simplification 
of  (A-ll)  will  produce 


2 

°g(x,Y) 


(Nnv  )2 
*0 


+ 


(NnY  )2 
*0 

(Nnv  )4 
0 


(Nnv  ) 
x0 


(Nnv  ) 
x0 


( A-2  3 ) 


which  becomes 


2(^>E{xiV-vV.  • 


(A-24) 


In  the  remaining  sections  some  results  for  the  three 
types  of  degradations  mentioned  above  will  be  presented 
and  discussed.  It  will  be  assumed  in  these  studies  that 
the  number  of  image  rows  is  large  (over  100).  This  is  not 
unreasonable  in  light  of  the  fact  that  all  images  used  in 
this  thesis  are  at  least  128x128  pixel  images. 


A. 4  Deletions 


Since  deletions  are  considered  independent  of  shifts 
and  insertions,  we  need  only  consider  the  expected  value 
of  (X/Y)  at  the  element  spacing  distance.  All  values  for 


(X/Y)  at  distances  other  than  the  element  spacing  or  its 
multiples  will  be  zero.  It  is  also  true  that  the  results 
will  be  the  same  for  different  element  spacing  distances 
provided  the  probability  that  an  edge  is  deleted  is  the 
same  and  the  number  of  edges  per  line  (given  no  deletions) 
is  the  same. 


If  C  is  the  number  of  edges  per  line  given  no 
deletions  then  the  expected  value  of  the  number  of  matches 
^t  the  element  spacing  distance  is  defined  as 


E-l 

=  ^  M*  P  (M)  , 
M=0 


(A-25 ) 


P(M)  *  P(M|D)P(D)  ,  ( A-26 ) 

D=0 

M  is  the  number  of  matches,  and  D  is  the  number  of 
deletions.  Since  each  edge  is  equally  likely  to  be 
deleted  we  have 

P(D)  =  <E)pD(l-p)E'D  ( A-27 ) 

for  each  D  where  p  is  the  probability  of  an  edge  being 
deleted.  P(M|D)  is  the  probability  that  the  number  of 
matches  is  M  given  D  deletions  occur.  This  number,  as 
well  as  P(CjD)  which  is  used  in  (A-30),  is  calculated  as 
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explained  in  the  outline  given  in  Section  A.l 
variance  is  defined  as 


The 


E-l 

£ 


(M-nY  )2P(m)  . 

*0 


Similarly, 


E-l 

y  c.p(o, 

c=o 


and 


E 

P(C)  =  P<CID>P<D>  * 

D=0 


E-l 


c=o 


(c-nv  )2P(c) 
o 


for  normalization  count,  C. 


(A-28) 


( A-29 ) 


( A-30 ) 


( A-3 1 ) 


It  remains  to  define  E{X^Y^}  to  provide  all  that  is 
necessary  for  the  computations  shown  in  (A-22)  and  (A-24). 
E  {X iY i  }  is  the  expected  value  of  the  product  of  the  number 
of  matches  and  normalization  count  for  a  given  line.  That 
is,  each  (X^,Y^)  pair  must  be  a  valid  pair  of  counts  for 
at  least  one  degraded  line  configuration  (or  for  the 
original  non-degraded  line).  (The  method  used  to 
calculate  E  fojYj) 


is  outlined  in  Section  A.l.)  Let  T  be 


the  set  of  valid  (number  of  matches,  normal iza tion  count) 
ordered  pairs.  Then 

E<xi.v  *  £  <Vj,p<xj'V  -  (*-32) 

(X^Y^eT 

where 

E 

PU^Yj)  =  ,Y_.)  1 D)  P  (D)  .  (A-33) 

D=0 

The  graph  in  Fig.  A-2  shows  the  expected  value  of 
(XA)  for  the  given  texture  period  as  a  function  of  the 
probability  of  deletion.  Only  the  1st  term  of  (A-22)  is 
used  since  the  second  term  is  insignificant  for  N  equal  to 
100  (which  was  used  in  this  example) .  The  expected  value 
of  (X/Y)  at  the  element  spacing  value  depends  only  on  the 
probability  that  an  edge  is  deleted.  The  number  of  edges 
per  line  and  the  element  spacing  distance  have  no  effect 
on  the  result.  However,  the  standard  deviation  decreases 
as  the  number  of  edges  per  line  increases.  For  100  image 
lines  the  standard  deviation;  is  less  than  one  tenth  of 
the  expected  value  for  probability  of  deletion  less  than 
or  equal  to  .5,  for  lines  with  at  least  5  edges.  The 
results  shown  in  Fig.  A-2  seem  to  indicate  that  if  no 
other  type  of  degradation  is  present  the  peak  at  the 
element  spacing  distance  will  be  classified  as  major  for 
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probability  of  deletion  as  high  as  .8. 

The  specific  probability  of  correct  classification  is 
not  given  because  the  threshold  used  in  the  classification 
scheme  is  dynamically  computed.  It  is  subject  to  the  same 
degradation  effects  as  the  peak  amplitudes.  Hence,  we  can 
only  use  these  results  to  get  an  indication  of  general 
trends. 

A. 5  Insertions 

In  the  case  of  insertions,  we  need  to  consider  the 
expected  value  of  the  element  spacing  peak,  as  well  as  the 
expected  values  at  the  surrounding  pixel  distances.  In 
this  way  the  expected  peak  to  valley  separation  can  be 
examined.  In  the  case  of  deletions  only  the  element 
spacing  ERA  value  (and  its  multiples)  could  be  expected  to 
be  non-zero.  The  initial,  non-degraded  line  configuration 
has  more  of  an  impact  in  the  case  of  insertions,  i.e.,  the 
length  of  the  line,  the  number  of  edges  per  line  and  the 
placement  of  these  edges  within  the  line  can  influence  the 
final  result. 

If  L  is  the  number  of  non-edge  pixels  on  a  line  given 
no  insertions  and  E  is  the  number  of  edges  on  the  line 
then  the  expected  value  of  the  number  of  matches  at 
distance,  D,  is  defined  as 
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( A-34 ) 


L+E-D 

y  m*p(m) 

M=0 


9 


P(M)  =  ^P(M|I)P(I)  ,  (A-35) 
1=0 

M  is  the  number  of  matches,  and  I  is  the  number  of 
insertions.  Since  each  non-edge  pixel  is  equally  likely 
to  become  an  insertion  we  have 


P(I)  =  (^)pI(l-p)  (L“I)  ,  ( A-36 ) 


for  each  I  where  p  is  the  probability  of  an  edge  being 
inserted  in  a  non-edge  location.  P (M | I )  is  the 
probability  that  the  number  of  matches  is  M  given  that  I 
insertions  occur.  The  variance  is  defined  as 

L+E-D 


z 


(M-n, 


)2p(m) 


( A-37 ) 


Similarly, 

L+E-D 

ny  =  ^  C-P(C)  ,  ( A-38 ) 

0  C=0 
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L 

P(C)  =  2>(C,I,P(I) 

1=0 


(A-39) 


and 


L+E-D 

(C-nv  )2P(C)  (A-40) 

*0 

P 

for  normalization  count  C. 


In  order  to  perform  the  computations  shown  in  (A-22) 
and  ( A-24 )  ,  E{XiYj.}  is  needed.  E{XjYj.}  is  the  expected 
value  of  the  product  of  the  number  of  matches  and 
normalization  count  for  a  given  line.  That  is,  each 
(X± / Y±)  pair  must  be  a  valid  pair  of  counts  for  at  least 
one  degraded  line  conf iguration  (or  for  the  original 
non-degraded  line).  Let  T  be  the  set  of  valid  (number  of 
matches,  normalization  count)  ordered  pairs.  Then 


EtX-Y.} 


,Y.)eT 


(Xj Yj  )  P  (Xj  ,  Yj  ) 


( A-4 1 ) 


where 


L 

P(Xj,Yj)  =  y;  P({X.,Y.)  1 1)  P  (I)  .  (A-4  2  ) 

1=0 

The  graphs  shown  in  Fig.  A-3  (a)  ,  ( b)  ,  and  (c)  show 
the  expected  value  of  (X/Y)  as  a  function  the  element 
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spacing  distance  and  the  probability  of  insertion  for 
specific  initial  non-degraded  line  configurations.  Only 
the  first  term  of  (A-22)  is  used  since  the  second  term  is 
insignificant  for  large  N.  (N  =  100  is  assumed.)  It  would 
seem  from  the  above  mentioned  graphs  that  length  of  line 
has  less  effect  on  expected  value  results  than  the  spacing 
between  valid  edges  in  the  line.  The  expected  value 
decreases  for  a  given  probability  of  insertions  as  the 
period  increases.  As  the  probability  of  insertions 
increases,  the  peak  valley  separation  for  the  element 
spacing  edge  repetition  array  decreases.  None  of  the 
graphs  shown  in  Fig.  A-3  would  pose  a  problem  in  terms  of 
major  peak  selection  since  the  values  shown  will  be 
multiplied  by  100.  However,  further  decreases  in  peak 
valley  separation  would  begin  to  have  an  affect. 

Assuming  a  one  hundred  line  image,  the  standard 
deviation  for  the  texture  period  peaks  and  their  multiples 
is  more  than  a  factor  of  ten  less  than  the  expected  values 
at  those  element  spacing  distances  for  all  cases  shown. 
As  the  length  of  the  line  increases  the  expected  value, 
standard  deviation  ratio  increases.  However,  for  a  given 
length  of  line,  as  the  element  period  increases  the  ratio 
decreases.  Also  the  standard  deviation  decreases  as  the 
square  of  the  number  of  image  rows. 
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Again,  no  precise  probabilities  can  be  given  for 
correct  classification.  In  this  case  not  only  do  the  peak 
classification  thresholds  depend  on  the  probability  of 
degradation,  but  the  peak  to  valley  separation  for  each 
graph  does  as  well. 

A. 6  Shifts 

For  this  study  is  is  assumed  that  each  edge  in  the 
image  line  has  a  non-edge  pixel  to  its  right  and  left  so 
that  each  edge  can  be  shifted  in  either  direction.  This 
was  done  for  ease  of  notation.  In  the  computed  cases,  the 
probability  that  an  edge  is  shifted  one  pixel  to  the  right 
is  the  same  as  the  probability  of  a  similar  left  shift. 
This  need  not  be  the  case  as  long  as  the  sum  of  the  two 
probabilities  does  not  exceed  1.0.  An  in  place  edge  is 
never  assumed  to  have  been  shifted  right  one  pixel  and 
then  shifted  left  this  amount.  Hence,  each  edge  can  only 
be  right  shifted,  left  shifted  or  unshifted.  In  the  case 
of  shifts,  as  for  insertions,  the  expected  values  for  the 
element  spacing  peak,  as  well  as  the  expected  values  at 
surrounding  pixel  distances  are  shown.  This  is  done  so 
that  the  expected  peak  to  valley  separations  can  be 
examined.  In  the  case  of  shifts,  the  expected  value  at 
the  texture  period  distance  is  the  same  for  different 
length  lines  and  different  texture  periods,  given  a 
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particular  left  shift,  right  shift  probability  pair.  (See 
Fig.  A-4(a),  (b),  and  (c).)  Only  the  first  term  of  (A-22) 
is  used  since  the  second  term  is  less  than  the  first  by  at 
least  a  factor  of  N.  (N  at  least  100  has  been  assumed.) 


Let  L  be  the  number  of  non-edge  pixels  within  the 
line  and  E  be  the  number  of  edges  in  the  line,  then  the 
expected  value  of  the  number  of  matches  at  distance,  D,  is 


L+E-D 

yi  m*p  (m) 

M=0 


( A-4  3 ) 


where 


P(M)  (M  |L,R)P(L,R)  ,  ( A-44 ) 

L,ReS 

L+R<E 

M  is  the  number  of  matches,  and  L(R)  is  the  number  of  left 
(right)  shifted  edges,  and  S  =  (0,1, ...,E}.  Since  each 
edge  is  equally  likely  to  be  left  shifted,  right  shifted 
or  remain  unshifted  we  have 

P(L,R)  =  (£)  (ERL)pLqR(1~P-<3)  (E_L“R)  ( A-45 ) 

for  each  L,R  pair  such  that  L  e  S,  R  e  S,  and  L+R  <  E, 
where  p  is  the  probability  of  an  edge  being  shifted  left, 
and  q  is  the  probability  of  an  edge  being  shifted  right. 
P(M|L,R)  is  the  probability  that  the  number  of  matches  is 
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M  given  L  left  shifts  and  R  right  shifts  occur  on  a 
previously  determined  line.  The  variance  is  defined  as 


Similarly, 


L+E-D 


C=0 


•P(C) 


r 


(A-46) 


(A-47) 


P(C)  =  S  (ciL'R>P<L'R>  ' 

L,  R£S 
L+R<E 

and , 


L+E-D 


12  <c-nY  >2P(C> 

c=o  0 


for  normalization  count  C. 


(A-48 ) 


( A-49 } 


^ ( X i )  will  now  be  defined.  E{X^Y^>  is  the  expected 
value  of  the  product  of  the  number  of  matches  and 
normalization  count  for  a  given  line.  That  is,  each 
Pair  must  be  a  valid  pair  of  counts  for  at  least 
one  degraded  line  configuration  (or  for  the  original 
non-degraded  line).  Let  T  be  the  set  of  valid  (number  of 
matches,  normalization  count)  ordered  pairs.  Then 


E{xiYi}  58  <xj*j)P(xj,Yj) 

(Xj , Yj ) £T 


(A-SO) 


where 


P(X.,Y.)  =  P((X..,Y..)  |  L,R)P  (L,R)  .  (A-51) 

L,  ReS 
L+R<E 

The  graphs  shown  in  Fig.  A-4  (a)  ,  (b)  ,  and  (c)  show 
the  expected  value  of  (XA)  as  a  function  of  p,  q,  the 
texture  period  for  specific  non-degraded  line 
configurations.  Again  only  the  first  term  of  (A-22)  is 
used.  From  the  graphic  results  it  seems  that  only  p  and  q 
exert  significant  influence  over  the  expected  value 
results.  When  p=q=(l/3)  the  peak-valley  separation  is 
close  to  the  point  at  which  the  texture  period  peak  will 
not  be  classified  as  major.  For  higher  probabilities  of 
shift  these  peaks  are  likely  to  fail  the  major  peak  test. 

As  the  length  of  the  line  increases  the  standard 
deviation  for  a  given  texture  period  decreases.  However, 
as  the  texture  period  increases  for  a  given  length  line, 
the  standard  deviation  increases.  For  the  texture  period 
results  shown,  the  standard  deviations  for  N  ■  100  are 
given  in  Table  A-l.  (That  is,  the  standard  deviations  are 
shown  only  for  the  texture  period  peaks.) 
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Figure  A-4(c).  Expected  Value  (E)  at  Distance  (D)  for 

Texture  Period  8 


TABLE  A-l.  EXPECTED  VALUE  AND  STANDARD  DEVIATION 
OF  (X/Y)  AT  THE  TEXTURE  PERIOD  FOR 
100  IMAGE  LINES 


Further  analysis  would  be  required  in  order  to 
determine  the  precise  probability  of  correctly  classifying 
texture  period  peaks.  This  is  due  to  the  dynamic 
thresholding  scheme  used  and  the  use  of  peak  to  valley 
separations. 
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